[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:37:22 dist-test-slave-tc2s krb5kdc[24893](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:37:22 dist-test-slave-tc2s krb5kdc[24893](info): set up 2 sockets
Aug 08 11:37:22 dist-test-slave-tc2s krb5kdc[24893](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 11:37:24.104270 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.403s user 0.006s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:37:24 dist-test-slave-tc2s krb5kdc[24893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:37:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:37:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
I20260808 11:37:24.297705 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:46855
--webserver_interface=127.24.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:43743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:46855
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:37:24.687987 24918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:37:24.688392 24918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:37:24.688548 24918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:37:24.695636 24918 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:37:24.695734 24918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:37:24.695812 24918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:37:24.695886 24918 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:37:24.695953 24918 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:37:24.705857 24918 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:43743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:46855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:46855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:37:24.707798 24918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:37:24.709939 24918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:37:24.733508 24926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:24.733860 24925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:24.733808 24929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:37:24.735054 24918 server_base.cc:1034] running on GCE node
I20260808 11:37:24.736192 24918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:37:24.738423 24918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:37:24.739635 24918 hybrid_clock.cc:648] HybridClock initialized: now 1786189044739532 us; error 96 us; skew 500 ppm
I20260808 11:37:24.740175 24918 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:37:24 dist-test-slave-tc2s krb5kdc[24893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189044, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:37:24.743649 24918 init.cc:377] Logged in from keytab as kudu/127.24.76.126@KRBTEST.COM (short username kudu)
I20260808 11:37:24.746088 24918 webserver.cc:460] Webserver started at http://127.24.76.126:33789/ using document root <none> and password file <none>
I20260808 11:37:24.747371 24918 fs_manager.cc:359] Metadata directory not provided
I20260808 11:37:24.747483 24918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:37:24.747932 24918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:37:24.752285 24918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "16e322c7e2a94477a3b172190b626e09"
format_stamp: "Formatted at 2026-08-08 11:37:24 on dist-test-slave-tc2s"
server_key: "4e3216e9719f78103ec659d1c9aca5c6"
server_key_iv: "2573158f6d61ff6d2ec9ef829628ee35"
server_key_version: "encryptionkey@0"
I20260808 11:37:24.753368 24918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "16e322c7e2a94477a3b172190b626e09"
format_stamp: "Formatted at 2026-08-08 11:37:24 on dist-test-slave-tc2s"
server_key: "4e3216e9719f78103ec659d1c9aca5c6"
server_key_iv: "2573158f6d61ff6d2ec9ef829628ee35"
server_key_version: "encryptionkey@0"
I20260808 11:37:24.761397 24918 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260808 11:37:24.767040 24937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:24.768942 24918 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260808 11:37:24.769199 24918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "16e322c7e2a94477a3b172190b626e09"
format_stamp: "Formatted at 2026-08-08 11:37:24 on dist-test-slave-tc2s"
server_key: "4e3216e9719f78103ec659d1c9aca5c6"
server_key_iv: "2573158f6d61ff6d2ec9ef829628ee35"
server_key_version: "encryptionkey@0"
I20260808 11:37:24.769408 24918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:37:24.790561 24918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:37:24.795783 24918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:37:24.796134 24918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:37:24.818852 24918 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:46855
I20260808 11:37:24.818861 25033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:46855 every 8 connection(s)
I20260808 11:37:24.821329 24918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:37:24.825047 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 24918
I20260808 11:37:24.825402 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:37:24.827656 24881 external_mini_cluster.cc:1428] Setting key 64183cc35bb5523a14ec73fbe3868fec
I20260808 11:37:24.828637 25037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:37:24.846271 25037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16e322c7e2a94477a3b172190b626e09: Bootstrap starting.
I20260808 11:37:24.852300 25037 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16e322c7e2a94477a3b172190b626e09: Neither blocks nor log segments found. Creating new log.
I20260808 11:37:24.853927 25037 log.cc:824] T 00000000000000000000000000000000 P 16e322c7e2a94477a3b172190b626e09: Log is configured to *not* fsync() on all Append() calls
I20260808 11:37:24.858124 25037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16e322c7e2a94477a3b172190b626e09: No bootstrap required, opened a new log
Aug 08 11:37:24 dist-test-slave-tc2s krb5kdc[24893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:37:24.870191 25037 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16e322c7e2a94477a3b172190b626e09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16e322c7e2a94477a3b172190b626e09" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 46855 } }
I20260808 11:37:24.870890 25037 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16e322c7e2a94477a3b172190b626e09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:37:24.871105 25037 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16e322c7e2a94477a3b172190b626e09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16e322c7e2a94477a3b172190b626e09, State: Initialized, Role: FOLLOWER
I20260808 11:37:24.872545 25037 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16e322c7e2a94477a3b172190b626e09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16e322c7e2a94477a3b172190b626e09" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 46855 } }
I20260808 11:37:24.873015 25037 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16e322c7e2a94477a3b172190b626e09 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:37:24.873385 25041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:24.833960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44524 (local address 127.24.76.126:46855)
0808 11:37:24.835068 (+ 1108us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:24.835081 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:24.836592 (+ 1511us) server_negotiation.cc:407] Connection header received
0808 11:37:24.845752 (+ 9160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:24.845773 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:24.846407 (+ 634us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:24.846826 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:24.850258 (+ 3432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:24.852144 (+ 1886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:24.854144 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:24.855236 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:24.855458 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:24.861475 (+ 6017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:24.861555 (+ 80us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:24.861609 (+ 54us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:24.861656 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:24.865975 (+ 4319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:24.867099 (+ 1124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:24.867108 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:24.867116 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:24.867408 (+ 292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:24.867929 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:24.867935 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:24.867938 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:24.868936 (+ 998us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:24.869351 (+ 415us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:24.870310 (+ 959us) server_negotiation.cc:299] Negotiation successful
0808 11:37:24.870751 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":748,"thread_start_us":376,"threads_started":1}
I20260808 11:37:24.873360 25037 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16e322c7e2a94477a3b172190b626e09 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:37:24.873664 25037 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16e322c7e2a94477a3b172190b626e09 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:37:24.876243 25037 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16e322c7e2a94477a3b172190b626e09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16e322c7e2a94477a3b172190b626e09" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 46855 } }
I20260808 11:37:24.877257 25037 leader_election.cc:302] T 00000000000000000000000000000000 P 16e322c7e2a94477a3b172190b626e09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16e322c7e2a94477a3b172190b626e09; no voters:
I20260808 11:37:24.878201 25037 leader_election.cc:288] T 00000000000000000000000000000000 P 16e322c7e2a94477a3b172190b626e09 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:37:24.878400 25046 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16e322c7e2a94477a3b172190b626e09 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:37:24.879009 25046 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16e322c7e2a94477a3b172190b626e09 [term 1 LEADER]: Becoming Leader. State: Replica: 16e322c7e2a94477a3b172190b626e09, State: Running, Role: LEADER
I20260808 11:37:24.879896 25046 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16e322c7e2a94477a3b172190b626e09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16e322c7e2a94477a3b172190b626e09" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 46855 } }
I20260808 11:37:24.882591 25037 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16e322c7e2a94477a3b172190b626e09 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:37:24.885286 25049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16e322c7e2a94477a3b172190b626e09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16e322c7e2a94477a3b172190b626e09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16e322c7e2a94477a3b172190b626e09" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 46855 } } }
I20260808 11:37:24.885164 25048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16e322c7e2a94477a3b172190b626e09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16e322c7e2a94477a3b172190b626e09. Latest consensus state: current_term: 1 leader_uuid: "16e322c7e2a94477a3b172190b626e09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16e322c7e2a94477a3b172190b626e09" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 46855 } } }
I20260808 11:37:24.887037 25049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16e322c7e2a94477a3b172190b626e09 [sys.catalog]: This master's current role is: LEADER
I20260808 11:37:24.887323 25048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16e322c7e2a94477a3b172190b626e09 [sys.catalog]: This master's current role is: LEADER
I20260808 11:37:24.894876 25061 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:37:24.901873 25061 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:37:24.915462 25061 catalog_manager.cc:1347] Generated new cluster ID: a8570a3661314ad5acb3770208808849
I20260808 11:37:24.915575 25061 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:37:24.939263 25061 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:37:24.940663 25061 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:37:24.950488 25061 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16e322c7e2a94477a3b172190b626e09: Generated new TSK 0
I20260808 11:37:24.951427 25061 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab.
Entry for principal kudu/127.24.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab.
I20260808 11:37:25.019776 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.65:0
--local_ip_for_outbound_sockets=127.24.76.65
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:46855
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:43743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:37:25.390128 25078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:37:25.390376 25078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:37:25.390508 25078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:37:25.397603 25078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:37:25.397711 25078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:37:25.397856 25078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.65
I20260808 11:37:25.407521 25078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:43743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:46855
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.65
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:37:25.408895 25078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:37:25.410549 25078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:37:25.423568 25086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:25.423842 25092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:25.423861 25088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:37:25.424564 25078 server_base.cc:1034] running on GCE node
I20260808 11:37:25.425151 25078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:37:25.426077 25078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:37:25.427273 25078 hybrid_clock.cc:648] HybridClock initialized: now 1786189045427224 us; error 39 us; skew 500 ppm
I20260808 11:37:25.427563 25078 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:37:25 dist-test-slave-tc2s krb5kdc[24893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189045, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:37:25.430629 25078 init.cc:377] Logged in from keytab as kudu/127.24.76.65@KRBTEST.COM (short username kudu)
I20260808 11:37:25.432037 25078 webserver.cc:460] Webserver started at http://127.24.76.65:36497/ using document root <none> and password file <none>
I20260808 11:37:25.432690 25078 fs_manager.cc:359] Metadata directory not provided
I20260808 11:37:25.432808 25078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:37:25.433104 25078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:37:25.436051 25078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-0/data/instance:
uuid: "7c121d2694e943d29d8ce8431cebb5d4"
format_stamp: "Formatted at 2026-08-08 11:37:25 on dist-test-slave-tc2s"
server_key: "df02f08b0a8a1ba0be4f51e030d79825"
server_key_iv: "398b68be46001c55848a0c570095c90d"
server_key_version: "encryptionkey@0"
I20260808 11:37:25.436774 25078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance:
uuid: "7c121d2694e943d29d8ce8431cebb5d4"
format_stamp: "Formatted at 2026-08-08 11:37:25 on dist-test-slave-tc2s"
server_key: "df02f08b0a8a1ba0be4f51e030d79825"
server_key_iv: "398b68be46001c55848a0c570095c90d"
server_key_version: "encryptionkey@0"
I20260808 11:37:25.441874 25078 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260808 11:37:25.445319 25110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:25.446628 25078 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 11:37:25.446774 25078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-0/wal
uuid: "7c121d2694e943d29d8ce8431cebb5d4"
format_stamp: "Formatted at 2026-08-08 11:37:25 on dist-test-slave-tc2s"
server_key: "df02f08b0a8a1ba0be4f51e030d79825"
server_key_iv: "398b68be46001c55848a0c570095c90d"
server_key_version: "encryptionkey@0"
I20260808 11:37:25.446947 25078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:37:25.461054 25078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:37:25.464955 25078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:37:25.465198 25078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:37:25.466326 25078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:37:25.467950 25078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:37:25.468027 25078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:25.468205 25078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:37:25.468274 25078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:25.503248 25078 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.65:44373
I20260808 11:37:25.503510 25294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.65:44373 every 8 connection(s)
I20260808 11:37:25.504592 25078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
I20260808 11:37:25.508370 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 25078
I20260808 11:37:25.508668 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance
I20260808 11:37:25.509552 24881 external_mini_cluster.cc:1428] Setting key f528daa120a0318a94657bca1afdb20f
Aug 08 11:37:25 dist-test-slave-tc2s krb5kdc[24893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189045, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.76.66@KRBTEST.COM; defaulting to no policy
I20260808 11:37:25.531615 25298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:25.508536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:58247 (local address 127.24.76.126:46855)
0808 11:37:25.509175 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:25.509182 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:25.510654 (+ 1472us) server_negotiation.cc:407] Connection header received
0808 11:37:25.513190 (+ 2536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:25.513196 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:25.513330 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:25.513618 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:25.515597 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:25.516679 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:25.518815 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:25.519584 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:25.519739 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:25.525091 (+ 5352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:25.525096 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:25.525099 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:25.525130 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:25.528003 (+ 2873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:25.529007 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:25.529011 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:25.529013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:25.529207 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:25.529750 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:25.529754 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:25.529756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:25.530057 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:25.530182 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:25.531033 (+ 851us) server_negotiation.cc:299] Negotiation successful
0808 11:37:25.531205 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":405,"threads_started":1}
I20260808 11:37:25.532606 25300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:25.509310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:46855 (local address 127.24.76.65:58247)
0808 11:37:25.510489 (+ 1179us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:25.510546 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:25.512920 (+ 2374us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:25.513827 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:25.513835 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:25.514565 (+ 730us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:37:25.515416 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:25.515437 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:25.516835 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:25.516839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:25.518655 (+ 1816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:25.518670 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:25.519915 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:25.519925 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:25.520364 (+ 439us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:25.521113 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:37:25.521140 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:37:25.524859 (+ 3719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:37:25.528147 (+ 3288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:25.528156 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:25.528171 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:25.528902 (+ 731us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:25.529325 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:25.529330 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:25.529334 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:25.529650 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:25.530247 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:37:25.530253 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:37:25.530566 (+ 313us) client_negotiation.cc:769] Sending connection context
0808 11:37:25.530853 (+ 287us) client_negotiation.cc:240] Negotiation successful
0808 11:37:25.531349 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":894,"thread_start_us":617,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.66@KRBTEST.COM" created.
I20260808 11:37:25.534588 25296 heartbeater.cc:347] Connected to a master server at 127.24.76.126:46855
I20260808 11:37:25.535064 25296 heartbeater.cc:464] Registering TS with master...
I20260808 11:37:25.536063 25296 heartbeater.cc:511] Master 127.24.76.126:46855 requested a full tablet report, sending...
I20260808 11:37:25.540071 24968 ts_manager.cc:194] Registered new tserver with Master: 7c121d2694e943d29d8ce8431cebb5d4 (127.24.76.65:44373)
I20260808 11:37:25.542416 24968 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.65@KRBTEST.COM'} at 127.24.76.65:58247
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab.
Entry for principal kudu/127.24.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab.
I20260808 11:37:25.559055 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.66:0
--local_ip_for_outbound_sockets=127.24.76.66
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:46855
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:43743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:37:25.990885 25304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:37:25.991153 25304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:37:25.991283 25304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:37:25.998893 25304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:37:25.998991 25304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:37:25.999140 25304 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.66
I20260808 11:37:26.008551 25304 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:43743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:46855
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.66
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:37:26.009742 25304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:37:26.011022 25304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:37:26.025409 25309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:26.025671 25310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:26.025974 25312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:37:26.027163 25304 server_base.cc:1034] running on GCE node
I20260808 11:37:26.027779 25304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:37:26.028930 25304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:37:26.030149 25304 hybrid_clock.cc:648] HybridClock initialized: now 1786189046030102 us; error 33 us; skew 500 ppm
I20260808 11:37:26.030604 25304 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:37:26 dist-test-slave-tc2s krb5kdc[24893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189046, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:37:26.034825 25304 init.cc:377] Logged in from keytab as kudu/127.24.76.66@KRBTEST.COM (short username kudu)
I20260808 11:37:26.036690 25304 webserver.cc:460] Webserver started at http://127.24.76.66:42273/ using document root <none> and password file <none>
I20260808 11:37:26.037495 25304 fs_manager.cc:359] Metadata directory not provided
I20260808 11:37:26.037645 25304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:37:26.038028 25304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:37:26.041195 25304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-1/data/instance:
uuid: "7f43d96c33ba413f8469faea7f9f7a92"
format_stamp: "Formatted at 2026-08-08 11:37:26 on dist-test-slave-tc2s"
server_key: "8251751dbc6e997e57cb192fea7571be"
server_key_iv: "a7a0072eda7cc5e8169c9838299ae954"
server_key_version: "encryptionkey@0"
I20260808 11:37:26.042126 25304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance:
uuid: "7f43d96c33ba413f8469faea7f9f7a92"
format_stamp: "Formatted at 2026-08-08 11:37:26 on dist-test-slave-tc2s"
server_key: "8251751dbc6e997e57cb192fea7571be"
server_key_iv: "a7a0072eda7cc5e8169c9838299ae954"
server_key_version: "encryptionkey@0"
I20260808 11:37:26.048112 25304 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 11:37:26.052167 25321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:26.053411 25304 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 11:37:26.053571 25304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-1/wal
uuid: "7f43d96c33ba413f8469faea7f9f7a92"
format_stamp: "Formatted at 2026-08-08 11:37:26 on dist-test-slave-tc2s"
server_key: "8251751dbc6e997e57cb192fea7571be"
server_key_iv: "a7a0072eda7cc5e8169c9838299ae954"
server_key_version: "encryptionkey@0"
I20260808 11:37:26.053795 25304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:37:26.092587 25304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:37:26.097411 25304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:37:26.097767 25304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:37:26.099093 25304 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:37:26.100893 25304 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:37:26.101019 25304 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:26.101156 25304 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:37:26.101245 25304 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:26.150230 25304 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.66:37401
I20260808 11:37:26.150480 25510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.66:37401 every 8 connection(s)
I20260808 11:37:26.151643 25304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
I20260808 11:37:26.157239 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 25304
I20260808 11:37:26.157460 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance
I20260808 11:37:26.158035 24881 external_mini_cluster.cc:1428] Setting key a87b5f379644b3547de13305c05f5b94
Aug 08 11:37:26 dist-test-slave-tc2s krb5kdc[24893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189046, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:37:26.174925 25519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:26.154828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:48009 (local address 127.24.76.126:46855)
0808 11:37:26.155280 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:26.155284 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:26.156567 (+ 1283us) server_negotiation.cc:407] Connection header received
0808 11:37:26.158943 (+ 2376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:26.158949 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:26.159035 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:26.159158 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:26.161652 (+ 2494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:26.162510 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:26.164230 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:26.164742 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:26.164857 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:26.168689 (+ 3832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:26.168694 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:26.168698 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:26.168730 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:26.171678 (+ 2948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:26.172417 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:26.172421 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:26.172424 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:26.172573 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:26.173036 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:26.173040 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:26.173041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:26.173337 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:26.173478 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:26.174320 (+ 842us) server_negotiation.cc:299] Negotiation successful
0808 11:37:26.174555 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":225,"threads_started":1}
I20260808 11:37:26.175849 25520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:26.155469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:46855 (local address 127.24.76.66:48009)
0808 11:37:26.156392 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:26.156440 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:26.158644 (+ 2204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:26.160222 (+ 1578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:26.160234 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:26.160851 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:37:26.161476 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:26.161494 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:26.162656 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:26.162661 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:26.164087 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:26.164100 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:26.164965 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:26.164972 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:26.165193 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:26.165939 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:37:26.165964 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:37:26.168481 (+ 2517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:37:26.171807 (+ 3326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:26.171816 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:26.171831 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:26.172304 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:26.172676 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:26.172681 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:26.172686 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:26.172926 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:26.173459 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:37:26.173467 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:37:26.173843 (+ 376us) client_negotiation.cc:769] Sending connection context
0808 11:37:26.174223 (+ 380us) client_negotiation.cc:240] Negotiation successful
0808 11:37:26.174724 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"thread_start_us":443,"threads_started":1}
WARNING: no policy specified for kudu/127.24.76.67@KRBTEST.COM; defaulting to no policy
I20260808 11:37:26.177354 25513 heartbeater.cc:347] Connected to a master server at 127.24.76.126:46855
I20260808 11:37:26.177696 25513 heartbeater.cc:464] Registering TS with master...
I20260808 11:37:26.178574 25513 heartbeater.cc:511] Master 127.24.76.126:46855 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.67@KRBTEST.COM" created.
I20260808 11:37:26.180155 24968 ts_manager.cc:194] Registered new tserver with Master: 7f43d96c33ba413f8469faea7f9f7a92 (127.24.76.66:37401)
I20260808 11:37:26.181231 24968 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.66@KRBTEST.COM'} at 127.24.76.66:48009
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab.
Entry for principal kudu/127.24.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab.
I20260808 11:37:26.203372 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.67:0
--local_ip_for_outbound_sockets=127.24.76.67
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:46855
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:43743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:37:26.546438 25296 heartbeater.cc:503] Master 127.24.76.126:46855 was elected leader, sending a full tablet report...
W20260808 11:37:26.622274 25526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:37:26.622596 25526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:37:26.622720 25526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:37:26.630905 25526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:37:26.631008 25526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:37:26.631129 25526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.67
I20260808 11:37:26.641817 25526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:43743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:46855
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.67
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:37:26.642969 25526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:37:26.644343 25526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:37:26.659227 25535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:26.659368 25538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:26.659366 25536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:37:26.660733 25526 server_base.cc:1034] running on GCE node
I20260808 11:37:26.661319 25526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:37:26.662349 25526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:37:26.663584 25526 hybrid_clock.cc:648] HybridClock initialized: now 1786189046663489 us; error 60 us; skew 500 ppm
I20260808 11:37:26.663895 25526 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:37:26 dist-test-slave-tc2s krb5kdc[24893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189046, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:37:26.671202 25526 init.cc:377] Logged in from keytab as kudu/127.24.76.67@KRBTEST.COM (short username kudu)
I20260808 11:37:26.672616 25526 webserver.cc:460] Webserver started at http://127.24.76.67:35633/ using document root <none> and password file <none>
I20260808 11:37:26.673293 25526 fs_manager.cc:359] Metadata directory not provided
I20260808 11:37:26.673429 25526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:37:26.673718 25526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:37:26.676995 25526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-2/data/instance:
uuid: "0c0c2088eba6428aba68a4377d60bbb3"
format_stamp: "Formatted at 2026-08-08 11:37:26 on dist-test-slave-tc2s"
server_key: "286620d22f90a60e227933e0c60b8ab3"
server_key_iv: "958305c5ee451135b494ba31fe8ca834"
server_key_version: "encryptionkey@0"
I20260808 11:37:26.677865 25526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance:
uuid: "0c0c2088eba6428aba68a4377d60bbb3"
format_stamp: "Formatted at 2026-08-08 11:37:26 on dist-test-slave-tc2s"
server_key: "286620d22f90a60e227933e0c60b8ab3"
server_key_iv: "958305c5ee451135b494ba31fe8ca834"
server_key_version: "encryptionkey@0"
I20260808 11:37:26.683799 25526 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 11:37:26.687902 25548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:26.689153 25526 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 11:37:26.689297 25526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-2/wal
uuid: "0c0c2088eba6428aba68a4377d60bbb3"
format_stamp: "Formatted at 2026-08-08 11:37:26 on dist-test-slave-tc2s"
server_key: "286620d22f90a60e227933e0c60b8ab3"
server_key_iv: "958305c5ee451135b494ba31fe8ca834"
server_key_version: "encryptionkey@0"
I20260808 11:37:26.689538 25526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:37:26.710966 25526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:37:26.715756 25526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:37:26.716112 25526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:37:26.717480 25526 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:37:26.719123 25526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:37:26.719276 25526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:26.719488 25526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:37:26.719574 25526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:26.773622 25526 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.67:43453
I20260808 11:37:26.773788 25717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.67:43453 every 8 connection(s)
I20260808 11:37:26.774979 25526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
I20260808 11:37:26.781589 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 25526
I20260808 11:37:26.781804 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance
I20260808 11:37:26.782384 24881 external_mini_cluster.cc:1428] Setting key 024c0af805ba8c24085319caec21a099
Aug 08 11:37:26 dist-test-slave-tc2s krb5kdc[24893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189046, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:37:26.799795 25722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:26.778385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:38451 (local address 127.24.76.126:46855)
0808 11:37:26.778938 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:26.778944 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:26.780401 (+ 1457us) server_negotiation.cc:407] Connection header received
0808 11:37:26.782876 (+ 2475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:26.782881 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:26.782988 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:26.783287 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:26.785391 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:26.786601 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:26.788378 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:26.788933 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:26.789029 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:26.792950 (+ 3921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:26.792954 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:26.792958 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:26.792990 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:26.796204 (+ 3214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:26.797002 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:26.797005 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:26.797008 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:26.797192 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:26.797742 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:26.797747 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:26.797750 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:26.798178 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:26.798399 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:26.799205 (+ 806us) server_negotiation.cc:299] Negotiation successful
0808 11:37:26.799417 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":279,"threads_started":1}
I20260808 11:37:26.800930 25725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:26.779109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:46855 (local address 127.24.76.67:38451)
0808 11:37:26.780085 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:26.780137 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:26.782603 (+ 2466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:26.783574 (+ 971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:26.783582 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:26.784386 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:37:26.785217 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:26.785245 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:26.786765 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:26.786770 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:26.788253 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:26.788266 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:26.789165 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:26.789172 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:26.789434 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:26.790063 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:37:26.790090 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:37:26.792760 (+ 2670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:37:26.796362 (+ 3602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:26.796369 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:26.796380 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:26.796903 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:26.797327 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:26.797330 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:26.797332 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:26.797633 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:26.798338 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:37:26.798344 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:37:26.798769 (+ 425us) client_negotiation.cc:769] Sending connection context
0808 11:37:26.799086 (+ 317us) client_negotiation.cc:240] Negotiation successful
0808 11:37:26.799542 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"thread_start_us":534,"threads_started":1}
I20260808 11:37:26.802496 25719 heartbeater.cc:347] Connected to a master server at 127.24.76.126:46855
I20260808 11:37:26.802850 25719 heartbeater.cc:464] Registering TS with master...
I20260808 11:37:26.803761 25719 heartbeater.cc:511] Master 127.24.76.126:46855 requested a full tablet report, sending...
I20260808 11:37:26.805207 24968 ts_manager.cc:194] Registered new tserver with Master: 0c0c2088eba6428aba68a4377d60bbb3 (127.24.76.67:43453)
I20260808 11:37:26.806017 24968 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.67@KRBTEST.COM'} at 127.24.76.67:38451
I20260808 11:37:26.811379 24881 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 11:37:26 dist-test-slave-tc2s krb5kdc[24893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189046, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 11:37:26 dist-test-slave-tc2s krb5kdc[24893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189046, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.76.65@KRBTEST.COM
I20260808 11:37:26.868896 25731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:26.852729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37020 (local address 127.24.76.65:44373)
0808 11:37:26.853575 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:26.853582 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:26.853609 (+ 27us) server_negotiation.cc:407] Connection header received
0808 11:37:26.853674 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:26.853681 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:26.853976 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:26.854106 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:26.855454 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:26.856771 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:26.858086 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:26.858914 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:26.859017 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:26.863196 (+ 4179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:26.863210 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:26.863225 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:26.863258 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:26.866412 (+ 3154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:26.866979 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:26.866985 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:26.866988 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:26.867153 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:26.867433 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:26.867436 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:26.867438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:26.867802 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:26.868025 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:26.868299 (+ 274us) server_negotiation.cc:299] Negotiation successful
0808 11:37:26.868543 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":477,"threads_started":1}
W20260808 11:37:26.870283 25154 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37020
Aug 08 11:37:26 dist-test-slave-tc2s krb5kdc[24893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 11:37:26 dist-test-slave-tc2s krb5kdc[24893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.65@KRBTEST.COM
I20260808 11:37:26.917677 25731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:26.905264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37022 (local address 127.24.76.65:44373)
0808 11:37:26.905485 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:26.905491 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:26.905563 (+ 72us) server_negotiation.cc:407] Connection header received
0808 11:37:26.905707 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:26.905711 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:26.905775 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:26.905856 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:26.907125 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:26.907788 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:26.908909 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:26.909345 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:26.909434 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:26.913162 (+ 3728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:26.913167 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:26.913171 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:26.913203 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:26.915638 (+ 2435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:26.916215 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:26.916219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:26.916221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:26.916347 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:26.916599 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:26.916602 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:26.916604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:26.916822 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:26.917005 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:26.917222 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 11:37:26.917383 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260808 11:37:26.919562 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 25078
I20260808 11:37:26.932031 25731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:26.920617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37034 (local address 127.24.76.65:44373)
0808 11:37:26.920759 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:26.920763 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:26.921071 (+ 308us) server_negotiation.cc:407] Connection header received
0808 11:37:26.921182 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:26.921185 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:26.921237 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:26.921307 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:26.922704 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:26.923415 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:26.924944 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:26.925363 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:26.925448 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:26.927443 (+ 1995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:26.927448 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:26.927453 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:26.927480 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:26.930055 (+ 2575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:26.930633 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:26.930636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:26.930638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:26.930755 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:26.931010 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:26.931013 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:26.931015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:26.931248 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:26.931421 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:26.931628 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 11:37:26.931770 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 11:37:26.932827 25288 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:37:27.019335 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 25078
I20260808 11:37:27.044919 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 25304
Aug 08 11:37:27 dist-test-slave-tc2s krb5kdc[24893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.66@KRBTEST.COM
I20260808 11:37:27.060483 25747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:27.045865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39148 (local address 127.24.76.66:37401)
0808 11:37:27.046622 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:27.046628 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:27.046655 (+ 27us) server_negotiation.cc:407] Connection header received
0808 11:37:27.046717 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:27.046724 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:27.046924 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:27.047054 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:27.048149 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:27.049151 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:27.050505 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:27.051044 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:27.051138 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:27.055194 (+ 4056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:27.055206 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:27.055220 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:27.055259 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:27.057920 (+ 2661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:27.058728 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:27.058737 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:27.058741 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:27.058882 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:27.059190 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:27.059193 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:27.059195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:27.059532 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:27.059748 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:27.059958 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 11:37:27.060153 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":385,"threads_started":1}
I20260808 11:37:27.061790 25501 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:37:27.143456 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 25304
I20260808 11:37:27.164788 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 25526
Aug 08 11:37:27 dist-test-slave-tc2s krb5kdc[24893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.67@KRBTEST.COM
I20260808 11:37:27.180101 25751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:27.165719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47654 (local address 127.24.76.67:43453)
0808 11:37:27.166369 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:27.166377 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:27.166405 (+ 28us) server_negotiation.cc:407] Connection header received
0808 11:37:27.166466 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:27.166472 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:27.166677 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:27.166800 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:27.167990 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:27.168991 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:27.170261 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:27.170809 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:27.170906 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:27.174962 (+ 4056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:27.174975 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:27.174989 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:27.175026 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:27.177564 (+ 2538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:27.178344 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:27.178352 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:27.178354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:27.178487 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:27.178762 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:27.178765 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:27.178767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:27.179100 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:27.179370 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:27.179537 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 11:37:27.179743 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":316,"threads_started":1}
I20260808 11:37:27.181454 25710 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:37:27.261801 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 25526
I20260808 11:37:27.285765 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 24918
I20260808 11:37:27.287014 25028 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:37:27.390388 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 24918
2026-08-08T11:37:27Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4755 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:37:27 dist-test-slave-tc2s krb5kdc[25756](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:37:27 dist-test-slave-tc2s krb5kdc[25756](info): set up 2 sockets
Aug 08 11:37:27 dist-test-slave-tc2s krb5kdc[25756](info): commencing operation
krb5kdc: starting...
W20260808 11:37:28.579345 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.000s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:37:28 dist-test-slave-tc2s krb5kdc[25756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:37:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:37:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
I20260808 11:37:28.753129 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:37591
--webserver_interface=127.24.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:36563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:37591
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:37:29.065302 25778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:37:29.065582 25778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:37:29.065711 25778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:37:29.073726 25778 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:37:29.073832 25778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:37:29.073928 25778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:37:29.074014 25778 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:37:29.074077 25778 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:37:29.083552 25778 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:36563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:37591
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:37591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:37:29.084743 25778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:37:29.086036 25778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:37:29.097606 25783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:29.097579 25786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:29.097658 25784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:37:29.098471 25778 server_base.cc:1034] running on GCE node
I20260808 11:37:29.099210 25778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:37:29.100521 25778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:37:29.101719 25778 hybrid_clock.cc:648] HybridClock initialized: now 1786189049101669 us; error 38 us; skew 500 ppm
I20260808 11:37:29.102094 25778 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:37:29 dist-test-slave-tc2s krb5kdc[25756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189049, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:37:29.105048 25778 init.cc:377] Logged in from keytab as kudu/127.24.76.126@KRBTEST.COM (short username kudu)
I20260808 11:37:29.106702 25778 webserver.cc:460] Webserver started at http://127.24.76.126:44531/ using document root <none> and password file <none>
I20260808 11:37:29.107366 25778 fs_manager.cc:359] Metadata directory not provided
I20260808 11:37:29.107486 25778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:37:29.107777 25778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:37:29.110700 25778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "bb22dddcc5544d51b853c2ef23bf1c39"
format_stamp: "Formatted at 2026-08-08 11:37:29 on dist-test-slave-tc2s"
server_key: "2562daffa95902cb8a61c7f62d14a7e5"
server_key_iv: "b89dfd7ef888b6f056a986b9f57a98f2"
server_key_version: "encryptionkey@0"
I20260808 11:37:29.111452 25778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "bb22dddcc5544d51b853c2ef23bf1c39"
format_stamp: "Formatted at 2026-08-08 11:37:29 on dist-test-slave-tc2s"
server_key: "2562daffa95902cb8a61c7f62d14a7e5"
server_key_iv: "b89dfd7ef888b6f056a986b9f57a98f2"
server_key_version: "encryptionkey@0"
I20260808 11:37:29.116365 25778 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 11:37:29.119626 25799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:29.120848 25778 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 11:37:29.121011 25778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "bb22dddcc5544d51b853c2ef23bf1c39"
format_stamp: "Formatted at 2026-08-08 11:37:29 on dist-test-slave-tc2s"
server_key: "2562daffa95902cb8a61c7f62d14a7e5"
server_key_iv: "b89dfd7ef888b6f056a986b9f57a98f2"
server_key_version: "encryptionkey@0"
I20260808 11:37:29.121181 25778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:37:29.139194 25778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:37:29.142800 25778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:37:29.143050 25778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:37:29.160081 25778 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:37591
I20260808 11:37:29.160104 25886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:37591 every 8 connection(s)
I20260808 11:37:29.161335 25778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:37:29.164923 25887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:37:29.167385 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 25778
I20260808 11:37:29.167620 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:37:29.168382 24881 external_mini_cluster.cc:1428] Setting key 0f48f0d5837328e1a04beddc073e8dcf
I20260808 11:37:29.176734 25887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb22dddcc5544d51b853c2ef23bf1c39: Bootstrap starting.
I20260808 11:37:29.180299 25887 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb22dddcc5544d51b853c2ef23bf1c39: Neither blocks nor log segments found. Creating new log.
I20260808 11:37:29.181522 25887 log.cc:824] T 00000000000000000000000000000000 P bb22dddcc5544d51b853c2ef23bf1c39: Log is configured to *not* fsync() on all Append() calls
Aug 08 11:37:29 dist-test-slave-tc2s krb5kdc[25756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:37:29.184366 25887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb22dddcc5544d51b853c2ef23bf1c39: No bootstrap required, opened a new log
I20260808 11:37:29.190554 25896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:29.170403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43958 (local address 127.24.76.126:37591)
0808 11:37:29.171114 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:29.171125 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:29.171164 (+ 39us) server_negotiation.cc:407] Connection header received
0808 11:37:29.172790 (+ 1626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:29.172811 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:29.173178 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:29.173502 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:29.175242 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:29.176506 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:29.178187 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:29.178854 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:29.178974 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:29.183631 (+ 4657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:29.183639 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:29.183653 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:29.183686 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:29.186664 (+ 2978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:29.187287 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:29.187293 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:29.187300 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:29.187474 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:29.187826 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:29.187830 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:29.187832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:29.188359 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:29.188658 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:29.189163 (+ 505us) server_negotiation.cc:299] Negotiation successful
0808 11:37:29.189538 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":316,"threads_started":1}
I20260808 11:37:29.192068 25887 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb22dddcc5544d51b853c2ef23bf1c39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb22dddcc5544d51b853c2ef23bf1c39" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 37591 } }
I20260808 11:37:29.192463 25887 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb22dddcc5544d51b853c2ef23bf1c39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:37:29.192587 25887 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb22dddcc5544d51b853c2ef23bf1c39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb22dddcc5544d51b853c2ef23bf1c39, State: Initialized, Role: FOLLOWER
I20260808 11:37:29.193187 25887 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb22dddcc5544d51b853c2ef23bf1c39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb22dddcc5544d51b853c2ef23bf1c39" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 37591 } }
I20260808 11:37:29.193481 25887 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb22dddcc5544d51b853c2ef23bf1c39 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:37:29.193609 25887 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb22dddcc5544d51b853c2ef23bf1c39 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:37:29.193761 25887 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb22dddcc5544d51b853c2ef23bf1c39 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:37:29.195887 25887 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb22dddcc5544d51b853c2ef23bf1c39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb22dddcc5544d51b853c2ef23bf1c39" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 37591 } }
I20260808 11:37:29.196436 25887 leader_election.cc:302] T 00000000000000000000000000000000 P bb22dddcc5544d51b853c2ef23bf1c39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb22dddcc5544d51b853c2ef23bf1c39; no voters:
I20260808 11:37:29.196985 25887 leader_election.cc:288] T 00000000000000000000000000000000 P bb22dddcc5544d51b853c2ef23bf1c39 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:37:29.197100 25899 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb22dddcc5544d51b853c2ef23bf1c39 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:37:29.197710 25899 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb22dddcc5544d51b853c2ef23bf1c39 [term 1 LEADER]: Becoming Leader. State: Replica: bb22dddcc5544d51b853c2ef23bf1c39, State: Running, Role: LEADER
I20260808 11:37:29.198215 25899 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb22dddcc5544d51b853c2ef23bf1c39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb22dddcc5544d51b853c2ef23bf1c39" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 37591 } }
I20260808 11:37:29.198953 25887 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb22dddcc5544d51b853c2ef23bf1c39 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:37:29.200881 25901 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb22dddcc5544d51b853c2ef23bf1c39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb22dddcc5544d51b853c2ef23bf1c39. Latest consensus state: current_term: 1 leader_uuid: "bb22dddcc5544d51b853c2ef23bf1c39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb22dddcc5544d51b853c2ef23bf1c39" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 37591 } } }
I20260808 11:37:29.201253 25901 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb22dddcc5544d51b853c2ef23bf1c39 [sys.catalog]: This master's current role is: LEADER
I20260808 11:37:29.201506 25900 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb22dddcc5544d51b853c2ef23bf1c39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb22dddcc5544d51b853c2ef23bf1c39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb22dddcc5544d51b853c2ef23bf1c39" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 37591 } } }
I20260808 11:37:29.201786 25900 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb22dddcc5544d51b853c2ef23bf1c39 [sys.catalog]: This master's current role is: LEADER
I20260808 11:37:29.206440 25909 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:37:29.211329 25909 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:37:29.218928 25909 catalog_manager.cc:1347] Generated new cluster ID: d5dc72af013342639c50968b74159fb4
I20260808 11:37:29.219035 25909 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:37:29.231140 25909 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:37:29.232931 25909 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:37:29.239271 25909 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb22dddcc5544d51b853c2ef23bf1c39: Generated new TSK 0
I20260808 11:37:29.240177 25909 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab.
Entry for principal kudu/127.24.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab.
I20260808 11:37:29.288625 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.65:0
--local_ip_for_outbound_sockets=127.24.76.65
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:37591
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:36563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:37:29.616083 25931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:37:29.616356 25931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:37:29.616494 25931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:37:29.623544 25931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:37:29.623642 25931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:37:29.623790 25931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.65
I20260808 11:37:29.633507 25931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:36563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:37591
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.65
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:37:29.634843 25931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:37:29.636191 25931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:37:29.649227 25941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:29.650928 25936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:29.651163 25939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:37:29.651525 25931 server_base.cc:1034] running on GCE node
I20260808 11:37:29.652155 25931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:37:29.653060 25931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:37:29.654254 25931 hybrid_clock.cc:648] HybridClock initialized: now 1786189049654209 us; error 45 us; skew 500 ppm
I20260808 11:37:29.654564 25931 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:37:29 dist-test-slave-tc2s krb5kdc[25756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189049, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:37:29.657562 25931 init.cc:377] Logged in from keytab as kudu/127.24.76.65@KRBTEST.COM (short username kudu)
I20260808 11:37:29.659058 25931 webserver.cc:460] Webserver started at http://127.24.76.65:45973/ using document root <none> and password file <none>
I20260808 11:37:29.659736 25931 fs_manager.cc:359] Metadata directory not provided
I20260808 11:37:29.659835 25931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:37:29.660095 25931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:37:29.662992 25931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-0/data/instance:
uuid: "63d0b32305d3467ca589af857a661212"
format_stamp: "Formatted at 2026-08-08 11:37:29 on dist-test-slave-tc2s"
server_key: "ad06ab5f8e608f6101033c0a3b12dc12"
server_key_iv: "c71d6268aa420cbe142857500a8a0dbc"
server_key_version: "encryptionkey@0"
I20260808 11:37:29.663728 25931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance:
uuid: "63d0b32305d3467ca589af857a661212"
format_stamp: "Formatted at 2026-08-08 11:37:29 on dist-test-slave-tc2s"
server_key: "ad06ab5f8e608f6101033c0a3b12dc12"
server_key_iv: "c71d6268aa420cbe142857500a8a0dbc"
server_key_version: "encryptionkey@0"
I20260808 11:37:29.668612 25931 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 11:37:29.671908 25950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:29.673118 25931 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 11:37:29.673244 25931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-0/wal
uuid: "63d0b32305d3467ca589af857a661212"
format_stamp: "Formatted at 2026-08-08 11:37:29 on dist-test-slave-tc2s"
server_key: "ad06ab5f8e608f6101033c0a3b12dc12"
server_key_iv: "c71d6268aa420cbe142857500a8a0dbc"
server_key_version: "encryptionkey@0"
I20260808 11:37:29.673400 25931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:37:29.708382 25931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:37:29.712281 25931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:37:29.712530 25931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:37:29.713433 25931 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:37:29.714784 25931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:37:29.714857 25931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:29.714955 25931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:37:29.714998 25931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:29.747063 25931 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.65:36549
I20260808 11:37:29.747112 26127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.65:36549 every 8 connection(s)
I20260808 11:37:29.748327 25931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
I20260808 11:37:29.756587 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 25931
I20260808 11:37:29.756803 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance
I20260808 11:37:29.757589 24881 external_mini_cluster.cc:1428] Setting key 872c8175a44aa54b2b2916201138f638
Aug 08 11:37:29 dist-test-slave-tc2s krb5kdc[25756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189049, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:37:29.768123 26133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:29.750861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:33513 (local address 127.24.76.126:37591)
0808 11:37:29.751329 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:29.751334 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:29.752257 (+ 923us) server_negotiation.cc:407] Connection header received
0808 11:37:29.754164 (+ 1907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:29.754170 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:29.754251 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:29.754352 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:29.755877 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:29.756598 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:29.758856 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:29.759294 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:29.759403 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:29.762823 (+ 3420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:29.762828 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:29.762831 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:29.762863 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:29.765145 (+ 2282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:29.765837 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:29.765842 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:29.765844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:29.765994 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:29.766408 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:29.766412 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:29.766414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:29.766680 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:29.766815 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:29.767538 (+ 723us) server_negotiation.cc:299] Negotiation successful
0808 11:37:29.767768 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":229,"threads_started":1}
I20260808 11:37:29.769098 26134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:29.751404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:37591 (local address 127.24.76.65:33513)
0808 11:37:29.752093 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:29.752136 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:29.753866 (+ 1730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:29.754602 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:29.754611 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:29.755122 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:37:29.755693 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:29.755711 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:29.756773 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:29.756780 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:29.758645 (+ 1865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:29.758661 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:29.759553 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:29.759562 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:29.759763 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:29.760462 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:37:29.760491 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:37:29.762609 (+ 2118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:37:29.765278 (+ 2669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:29.765287 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:29.765304 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:29.765710 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:29.766099 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:29.766103 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:29.766107 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:29.766291 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:29.766808 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:37:29.766816 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:37:29.767152 (+ 336us) client_negotiation.cc:769] Sending connection context
0808 11:37:29.767387 (+ 235us) client_negotiation.cc:240] Negotiation successful
0808 11:37:29.767911 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":264,"threads_started":1}
I20260808 11:37:29.770651 26128 heartbeater.cc:347] Connected to a master server at 127.24.76.126:37591
I20260808 11:37:29.771041 26128 heartbeater.cc:464] Registering TS with master...
I20260808 11:37:29.771879 26128 heartbeater.cc:511] Master 127.24.76.126:37591 requested a full tablet report, sending...
I20260808 11:37:29.774284 25826 ts_manager.cc:194] Registered new tserver with Master: 63d0b32305d3467ca589af857a661212 (127.24.76.65:36549)
I20260808 11:37:29.775840 25826 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.65@KRBTEST.COM'} at 127.24.76.65:33513
WARNING: no policy specified for kudu/127.24.76.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab.
Entry for principal kudu/127.24.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab.
I20260808 11:37:29.805620 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.66:0
--local_ip_for_outbound_sockets=127.24.76.66
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:37591
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:36563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:37:30.124513 26139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:37:30.124797 26139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:37:30.124933 26139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:37:30.132036 26139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:37:30.132131 26139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:37:30.132282 26139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.66
I20260808 11:37:30.141723 26139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:36563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:37591
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.66
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:37:30.142845 26139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:37:30.144090 26139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:37:30.155928 26147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:30.156020 26145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:30.155977 26149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:37:30.156935 26139 server_base.cc:1034] running on GCE node
I20260808 11:37:30.157485 26139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:37:30.158349 26139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:37:30.159540 26139 hybrid_clock.cc:648] HybridClock initialized: now 1786189050159495 us; error 39 us; skew 500 ppm
I20260808 11:37:30.159852 26139 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:37:30 dist-test-slave-tc2s krb5kdc[25756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189050, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:37:30.162875 26139 init.cc:377] Logged in from keytab as kudu/127.24.76.66@KRBTEST.COM (short username kudu)
I20260808 11:37:30.171232 26139 webserver.cc:460] Webserver started at http://127.24.76.66:33741/ using document root <none> and password file <none>
I20260808 11:37:30.171901 26139 fs_manager.cc:359] Metadata directory not provided
I20260808 11:37:30.172039 26139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:37:30.172319 26139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:37:30.175066 26139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-1/data/instance:
uuid: "d11b25dc5a734c0597109a22dfefc73f"
format_stamp: "Formatted at 2026-08-08 11:37:30 on dist-test-slave-tc2s"
server_key: "caab2f07853a8da690585d72cc298a91"
server_key_iv: "659c902eca46ca58a773b09e88467587"
server_key_version: "encryptionkey@0"
I20260808 11:37:30.175796 26139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance:
uuid: "d11b25dc5a734c0597109a22dfefc73f"
format_stamp: "Formatted at 2026-08-08 11:37:30 on dist-test-slave-tc2s"
server_key: "caab2f07853a8da690585d72cc298a91"
server_key_iv: "659c902eca46ca58a773b09e88467587"
server_key_version: "encryptionkey@0"
I20260808 11:37:30.180889 26139 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 11:37:30.184209 26167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:30.185416 26139 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 11:37:30.185586 26139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-1/wal
uuid: "d11b25dc5a734c0597109a22dfefc73f"
format_stamp: "Formatted at 2026-08-08 11:37:30 on dist-test-slave-tc2s"
server_key: "caab2f07853a8da690585d72cc298a91"
server_key_iv: "659c902eca46ca58a773b09e88467587"
server_key_version: "encryptionkey@0"
I20260808 11:37:30.185756 26139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:37:30.198248 26139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:37:30.202098 26139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:37:30.202342 26139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:37:30.203181 26139 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:37:30.204427 26139 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:37:30.204497 26139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:30.204603 26139 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:37:30.204654 26139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:30.232685 26139 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.66:44351
I20260808 11:37:30.232738 26350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.66:44351 every 8 connection(s)
I20260808 11:37:30.233848 26139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
I20260808 11:37:30.243175 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 26139
I20260808 11:37:30.243350 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance
I20260808 11:37:30.243753 24881 external_mini_cluster.cc:1428] Setting key e081052daf10a78cba727758e603a0bb
Aug 08 11:37:30 dist-test-slave-tc2s krb5kdc[25756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189050, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:37:30.252512 26133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:30.236291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:42285 (local address 127.24.76.126:37591)
0808 11:37:30.236469 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:30.236474 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:30.237547 (+ 1073us) server_negotiation.cc:407] Connection header received
0808 11:37:30.239390 (+ 1843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:30.239396 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:30.239471 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:30.239565 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:30.241001 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:30.241714 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:30.243243 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:30.243732 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:30.243853 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:30.247203 (+ 3350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:30.247208 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:30.247211 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:30.247243 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:30.249589 (+ 2346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:30.250338 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:30.250342 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:30.250345 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:30.250468 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:30.250877 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:30.250881 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:30.250883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:30.251167 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:30.251299 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:30.251960 (+ 661us) server_negotiation.cc:299] Negotiation successful
0808 11:37:30.252194 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 11:37:30.253362 26353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:30.236712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:37591 (local address 127.24.76.66:42285)
0808 11:37:30.237389 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:30.237434 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:30.239114 (+ 1680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:30.239806 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:30.239814 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:30.240275 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:37:30.240831 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:30.240848 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:30.241851 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:30.241855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:30.243105 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:30.243118 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:30.243945 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:30.243953 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:30.244163 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:30.244842 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:37:30.244872 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:37:30.246999 (+ 2127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:37:30.249725 (+ 2726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:30.249734 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:30.249749 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:30.250218 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:30.250573 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:30.250578 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:30.250582 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:30.250763 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:30.251290 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:37:30.251298 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:37:30.251606 (+ 308us) client_negotiation.cc:769] Sending connection context
0808 11:37:30.251826 (+ 220us) client_negotiation.cc:240] Negotiation successful
0808 11:37:30.252297 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":271,"threads_started":1}
I20260808 11:37:30.254669 26351 heartbeater.cc:347] Connected to a master server at 127.24.76.126:37591
I20260808 11:37:30.254937 26351 heartbeater.cc:464] Registering TS with master...
I20260808 11:37:30.255630 26351 heartbeater.cc:511] Master 127.24.76.126:37591 requested a full tablet report, sending...
I20260808 11:37:30.257015 25826 ts_manager.cc:194] Registered new tserver with Master: d11b25dc5a734c0597109a22dfefc73f (127.24.76.66:44351)
I20260808 11:37:30.258045 25826 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.66@KRBTEST.COM'} at 127.24.76.66:42285
WARNING: no policy specified for kudu/127.24.76.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab.
Entry for principal kudu/127.24.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab.
I20260808 11:37:30.285030 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.67:0
--local_ip_for_outbound_sockets=127.24.76.67
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:37591
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:36563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:37:30.603991 26362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:37:30.604250 26362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:37:30.604384 26362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:37:30.611452 26362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:37:30.611548 26362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:37:30.611694 26362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.67
I20260808 11:37:30.621031 26362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:36563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:37591
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.67
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:37:30.622223 26362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:37:30.623502 26362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:37:30.635586 26373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:30.635702 26370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:30.635638 26371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:37:30.636698 26362 server_base.cc:1034] running on GCE node
I20260808 11:37:30.637218 26362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:37:30.638063 26362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:37:30.639259 26362 hybrid_clock.cc:648] HybridClock initialized: now 1786189050639236 us; error 68 us; skew 500 ppm
I20260808 11:37:30.639539 26362 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:37:30 dist-test-slave-tc2s krb5kdc[25756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189050, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:37:30.642504 26362 init.cc:377] Logged in from keytab as kudu/127.24.76.67@KRBTEST.COM (short username kudu)
I20260808 11:37:30.643766 26362 webserver.cc:460] Webserver started at http://127.24.76.67:41139/ using document root <none> and password file <none>
I20260808 11:37:30.644387 26362 fs_manager.cc:359] Metadata directory not provided
I20260808 11:37:30.644498 26362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:37:30.644786 26362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:37:30.647583 26362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-2/data/instance:
uuid: "149c0c323ccd4db383423eb5caa9b8c2"
format_stamp: "Formatted at 2026-08-08 11:37:30 on dist-test-slave-tc2s"
server_key: "b112014df4c55f2a39b3c2ddf9a60a6b"
server_key_iv: "b2924a99671aee8a09ac5aa771b7eacb"
server_key_version: "encryptionkey@0"
I20260808 11:37:30.648357 26362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance:
uuid: "149c0c323ccd4db383423eb5caa9b8c2"
format_stamp: "Formatted at 2026-08-08 11:37:30 on dist-test-slave-tc2s"
server_key: "b112014df4c55f2a39b3c2ddf9a60a6b"
server_key_iv: "b2924a99671aee8a09ac5aa771b7eacb"
server_key_version: "encryptionkey@0"
I20260808 11:37:30.653167 26362 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 11:37:30.656360 26382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:30.657514 26362 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 11:37:30.657655 26362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-2/wal
uuid: "149c0c323ccd4db383423eb5caa9b8c2"
format_stamp: "Formatted at 2026-08-08 11:37:30 on dist-test-slave-tc2s"
server_key: "b112014df4c55f2a39b3c2ddf9a60a6b"
server_key_iv: "b2924a99671aee8a09ac5aa771b7eacb"
server_key_version: "encryptionkey@0"
I20260808 11:37:30.657823 26362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:37:30.668378 26362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:37:30.672152 26362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:37:30.672405 26362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:37:30.673223 26362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:37:30.674439 26362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:37:30.674516 26362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:30.674630 26362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:37:30.674685 26362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:30.707332 26362 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.67:41257
I20260808 11:37:30.707324 26545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.67:41257 every 8 connection(s)
I20260808 11:37:30.708660 26362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
I20260808 11:37:30.709769 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 26362
I20260808 11:37:30.709990 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance
I20260808 11:37:30.710476 24881 external_mini_cluster.cc:1428] Setting key 9b382b67deef75001399e8f7d38c2041
Aug 08 11:37:30 dist-test-slave-tc2s krb5kdc[25756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189050, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:37:30.729260 26133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:30.711578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:34781 (local address 127.24.76.126:37591)
0808 11:37:30.711744 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:30.711750 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:30.713112 (+ 1362us) server_negotiation.cc:407] Connection header received
0808 11:37:30.715262 (+ 2150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:30.715268 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:30.715400 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:30.715496 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:30.717091 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:30.717836 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:30.719361 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:30.719772 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:30.719876 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:30.723453 (+ 3577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:30.723457 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:30.723461 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:30.723499 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:30.726462 (+ 2963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:30.727076 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:30.727080 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:30.727082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:30.727248 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:30.727648 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:30.727651 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:30.727653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:30.727940 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:30.728079 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:30.728703 (+ 624us) server_negotiation.cc:299] Negotiation successful
0808 11:37:30.728851 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 11:37:30.730080 26549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:30.712151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:37591 (local address 127.24.76.67:34781)
0808 11:37:30.712970 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:30.713014 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:30.715019 (+ 2005us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:30.715736 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:30.715743 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:30.716302 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:37:30.716925 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:30.716945 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:30.717999 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:30.718005 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:30.719223 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:30.719236 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:30.719994 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:30.720002 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:30.720231 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:30.720866 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:37:30.720887 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:37:30.723267 (+ 2380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:37:30.726575 (+ 3308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:30.726582 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:30.726592 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:30.726973 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:30.727328 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:30.727331 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:30.727333 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:30.727568 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:30.728062 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:37:30.728067 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:37:30.728363 (+ 296us) client_negotiation.cc:769] Sending connection context
0808 11:37:30.728586 (+ 223us) client_negotiation.cc:240] Negotiation successful
0808 11:37:30.728950 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":400,"threads_started":1}
I20260808 11:37:30.731215 26546 heartbeater.cc:347] Connected to a master server at 127.24.76.126:37591
I20260808 11:37:30.731498 26546 heartbeater.cc:464] Registering TS with master...
I20260808 11:37:30.732447 26546 heartbeater.cc:511] Master 127.24.76.126:37591 requested a full tablet report, sending...
I20260808 11:37:30.733976 25826 ts_manager.cc:194] Registered new tserver with Master: 149c0c323ccd4db383423eb5caa9b8c2 (127.24.76.67:41257)
I20260808 11:37:30.734762 25826 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.67@KRBTEST.COM'} at 127.24.76.67:34781
I20260808 11:37:30.737707 24881 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:37:30.760185 26133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:30.745239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43974 (local address 127.24.76.126:37591)
0808 11:37:30.745396 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:30.745400 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:30.746179 (+ 779us) server_negotiation.cc:407] Connection header received
0808 11:37:30.746393 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:30.746398 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:30.746456 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:30.746538 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:30.749140 (+ 2602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:30.750001 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:30.751917 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:30.752399 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:30.752490 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:30.754964 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:30.754968 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:30.754971 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:30.755000 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:30.757756 (+ 2756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:30.758313 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:30.758316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:30.758318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:30.758481 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:30.758762 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:30.758765 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:30.758767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:30.759093 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:30.759303 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:30.759624 (+ 321us) server_negotiation.cc:299] Negotiation successful
0808 11:37:30.759825 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 11:37:30.768237 25826 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43974:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:37:30.777601 25826 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:37:30.778975 26128 heartbeater.cc:503] Master 127.24.76.126:37591 was elected leader, sending a full tablet report...
I20260808 11:37:30.819569 26568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:30.795915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:36549 (local address 127.0.0.1:37802)
0808 11:37:30.810090 (+ 14175us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:30.810112 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:30.810249 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:30.811671 (+ 1422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:30.811675 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:30.811702 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:37:30.812288 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:30.812302 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:30.814478 (+ 2176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:30.814483 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:30.816711 (+ 2228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:30.816724 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:30.818448 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:30.818455 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:30.818656 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:30.818712 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 11:37:30.818838 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 11:37:30.819060 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14034,"thread_start_us":14434,"threads_started":1}
I20260808 11:37:30.819764 26561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:30.793777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:44351 (local address 127.0.0.1:46992)
0808 11:37:30.794745 (+ 968us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:30.794786 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:30.794985 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:30.812365 (+ 17380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:30.812369 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:30.812394 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:37:30.812888 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:30.812905 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:30.814926 (+ 2021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:30.814932 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:30.816840 (+ 1908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:30.816858 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:30.818898 (+ 2040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:30.818904 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:30.819077 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:30.819111 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 11:37:30.819217 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 11:37:30.819351 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":713,"thread_start_us":1303,"threads_started":1}
I20260808 11:37:30.820343 26562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:30.793997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46992 (local address 127.24.76.66:44351)
0808 11:37:30.811730 (+ 17733us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:30.811738 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:30.811766 (+ 28us) server_negotiation.cc:407] Connection header received
0808 11:37:30.811856 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:30.811863 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:30.812126 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:30.812351 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:37:30.813094 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:30.814738 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:30.817014 (+ 2276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:30.818674 (+ 1660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:30.818813 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:30.819009 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:30.819274 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 11:37:30.819438 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17586,"thread_start_us":17362,"threads_started":1}
I20260808 11:37:30.820937 26564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:30.794447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:41257 (local address 127.0.0.1:50480)
0808 11:37:30.795630 (+ 1183us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:30.795646 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:30.795752 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:30.810923 (+ 15171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:30.810930 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:30.810963 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:37:30.811468 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:30.811484 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:30.813659 (+ 2175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:30.813665 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:30.818060 (+ 4395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:30.818076 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:30.820225 (+ 2149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:30.820231 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:30.820485 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:30.820520 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 11:37:30.820608 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 11:37:30.820707 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1045,"thread_start_us":896,"threads_started":1}
I20260808 11:37:30.820950 26569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:30.796312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37802 (local address 127.24.76.65:36549)
0808 11:37:30.811093 (+ 14781us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:30.811101 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:30.811129 (+ 28us) server_negotiation.cc:407] Connection header received
0808 11:37:30.811223 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:30.811231 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:30.811487 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:30.811663 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:37:30.812488 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:30.814330 (+ 1842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:30.816876 (+ 2546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:30.818236 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:30.818382 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:30.818617 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:30.820010 (+ 1393us) server_negotiation.cc:299] Negotiation successful
0808 11:37:30.820238 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14612,"thread_start_us":14358,"threads_started":1}
I20260808 11:37:30.821684 26565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:30.794620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50480 (local address 127.24.76.67:41257)
0808 11:37:30.810207 (+ 15587us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:30.810218 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:30.810261 (+ 43us) server_negotiation.cc:407] Connection header received
0808 11:37:30.810378 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:30.810386 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:30.810670 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:30.810909 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:37:30.811803 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:30.813504 (+ 1701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:30.818272 (+ 4768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:30.819784 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:30.819909 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:30.820102 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:30.821294 (+ 1192us) server_negotiation.cc:299] Negotiation successful
0808 11:37:30.821430 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15406,"thread_start_us":14982,"threads_started":1}
I20260808 11:37:30.825395 26458 tablet_service.cc:1467] Processing CreateTablet for tablet 8f7905fc4ba74a51a23d158937f688c1 (DEFAULT_TABLE table=test-table [id=0d834a864f2f4a1ca6ba9af6b6b9d506]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:37:30.825237 26246 tablet_service.cc:1467] Processing CreateTablet for tablet 8f7905fc4ba74a51a23d158937f688c1 (DEFAULT_TABLE table=test-table [id=0d834a864f2f4a1ca6ba9af6b6b9d506]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:37:30.826907 26246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f7905fc4ba74a51a23d158937f688c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:37:30.826828 26027 tablet_service.cc:1467] Processing CreateTablet for tablet 8f7905fc4ba74a51a23d158937f688c1 (DEFAULT_TABLE table=test-table [id=0d834a864f2f4a1ca6ba9af6b6b9d506]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:37:30.827952 26027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f7905fc4ba74a51a23d158937f688c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:37:30.828605 26458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f7905fc4ba74a51a23d158937f688c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:37:30.840807 26570 tablet_bootstrap.cc:492] T 8f7905fc4ba74a51a23d158937f688c1 P d11b25dc5a734c0597109a22dfefc73f: Bootstrap starting.
I20260808 11:37:30.843926 26571 tablet_bootstrap.cc:492] T 8f7905fc4ba74a51a23d158937f688c1 P 63d0b32305d3467ca589af857a661212: Bootstrap starting.
I20260808 11:37:30.845467 26570 tablet_bootstrap.cc:654] T 8f7905fc4ba74a51a23d158937f688c1 P d11b25dc5a734c0597109a22dfefc73f: Neither blocks nor log segments found. Creating new log.
I20260808 11:37:30.846223 26573 tablet_bootstrap.cc:492] T 8f7905fc4ba74a51a23d158937f688c1 P 149c0c323ccd4db383423eb5caa9b8c2: Bootstrap starting.
I20260808 11:37:30.847172 26570 log.cc:824] T 8f7905fc4ba74a51a23d158937f688c1 P d11b25dc5a734c0597109a22dfefc73f: Log is configured to *not* fsync() on all Append() calls
I20260808 11:37:30.848395 26571 tablet_bootstrap.cc:654] T 8f7905fc4ba74a51a23d158937f688c1 P 63d0b32305d3467ca589af857a661212: Neither blocks nor log segments found. Creating new log.
I20260808 11:37:30.849570 26573 tablet_bootstrap.cc:654] T 8f7905fc4ba74a51a23d158937f688c1 P 149c0c323ccd4db383423eb5caa9b8c2: Neither blocks nor log segments found. Creating new log.
I20260808 11:37:30.849843 26571 log.cc:824] T 8f7905fc4ba74a51a23d158937f688c1 P 63d0b32305d3467ca589af857a661212: Log is configured to *not* fsync() on all Append() calls
I20260808 11:37:30.850860 26570 tablet_bootstrap.cc:492] T 8f7905fc4ba74a51a23d158937f688c1 P d11b25dc5a734c0597109a22dfefc73f: No bootstrap required, opened a new log
I20260808 11:37:30.851087 26570 ts_tablet_manager.cc:1397] T 8f7905fc4ba74a51a23d158937f688c1 P d11b25dc5a734c0597109a22dfefc73f: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.007s
I20260808 11:37:30.851226 26573 log.cc:824] T 8f7905fc4ba74a51a23d158937f688c1 P 149c0c323ccd4db383423eb5caa9b8c2: Log is configured to *not* fsync() on all Append() calls
I20260808 11:37:30.853349 26571 tablet_bootstrap.cc:492] T 8f7905fc4ba74a51a23d158937f688c1 P 63d0b32305d3467ca589af857a661212: No bootstrap required, opened a new log
I20260808 11:37:30.853644 26571 ts_tablet_manager.cc:1397] T 8f7905fc4ba74a51a23d158937f688c1 P 63d0b32305d3467ca589af857a661212: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260808 11:37:30.854141 26573 tablet_bootstrap.cc:492] T 8f7905fc4ba74a51a23d158937f688c1 P 149c0c323ccd4db383423eb5caa9b8c2: No bootstrap required, opened a new log
I20260808 11:37:30.854432 26573 ts_tablet_manager.cc:1397] T 8f7905fc4ba74a51a23d158937f688c1 P 149c0c323ccd4db383423eb5caa9b8c2: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260808 11:37:30.863114 26573 raft_consensus.cc:348] T 8f7905fc4ba74a51a23d158937f688c1 P 149c0c323ccd4db383423eb5caa9b8c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d11b25dc5a734c0597109a22dfefc73f" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44351 } } peers { permanent_uuid: "149c0c323ccd4db383423eb5caa9b8c2" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41257 } } peers { permanent_uuid: "63d0b32305d3467ca589af857a661212" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 36549 } }
I20260808 11:37:30.863567 26573 raft_consensus.cc:374] T 8f7905fc4ba74a51a23d158937f688c1 P 149c0c323ccd4db383423eb5caa9b8c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:37:30.863673 26573 raft_consensus.cc:729] T 8f7905fc4ba74a51a23d158937f688c1 P 149c0c323ccd4db383423eb5caa9b8c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 149c0c323ccd4db383423eb5caa9b8c2, State: Initialized, Role: FOLLOWER
I20260808 11:37:30.864253 26573 consensus_queue.cc:260] T 8f7905fc4ba74a51a23d158937f688c1 P 149c0c323ccd4db383423eb5caa9b8c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d11b25dc5a734c0597109a22dfefc73f" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44351 } } peers { permanent_uuid: "149c0c323ccd4db383423eb5caa9b8c2" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41257 } } peers { permanent_uuid: "63d0b32305d3467ca589af857a661212" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 36549 } }
I20260808 11:37:30.864434 26570 raft_consensus.cc:348] T 8f7905fc4ba74a51a23d158937f688c1 P d11b25dc5a734c0597109a22dfefc73f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d11b25dc5a734c0597109a22dfefc73f" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44351 } } peers { permanent_uuid: "149c0c323ccd4db383423eb5caa9b8c2" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41257 } } peers { permanent_uuid: "63d0b32305d3467ca589af857a661212" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 36549 } }
I20260808 11:37:30.864859 26570 raft_consensus.cc:374] T 8f7905fc4ba74a51a23d158937f688c1 P d11b25dc5a734c0597109a22dfefc73f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:37:30.865031 26570 raft_consensus.cc:729] T 8f7905fc4ba74a51a23d158937f688c1 P d11b25dc5a734c0597109a22dfefc73f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d11b25dc5a734c0597109a22dfefc73f, State: Initialized, Role: FOLLOWER
I20260808 11:37:30.865402 26571 raft_consensus.cc:348] T 8f7905fc4ba74a51a23d158937f688c1 P 63d0b32305d3467ca589af857a661212 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d11b25dc5a734c0597109a22dfefc73f" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44351 } } peers { permanent_uuid: "149c0c323ccd4db383423eb5caa9b8c2" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41257 } } peers { permanent_uuid: "63d0b32305d3467ca589af857a661212" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 36549 } }
I20260808 11:37:30.865669 26546 heartbeater.cc:503] Master 127.24.76.126:37591 was elected leader, sending a full tablet report...
I20260808 11:37:30.865829 26571 raft_consensus.cc:374] T 8f7905fc4ba74a51a23d158937f688c1 P 63d0b32305d3467ca589af857a661212 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:37:30.866000 26571 raft_consensus.cc:729] T 8f7905fc4ba74a51a23d158937f688c1 P 63d0b32305d3467ca589af857a661212 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63d0b32305d3467ca589af857a661212, State: Initialized, Role: FOLLOWER
I20260808 11:37:30.865847 26570 consensus_queue.cc:260] T 8f7905fc4ba74a51a23d158937f688c1 P d11b25dc5a734c0597109a22dfefc73f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d11b25dc5a734c0597109a22dfefc73f" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44351 } } peers { permanent_uuid: "149c0c323ccd4db383423eb5caa9b8c2" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41257 } } peers { permanent_uuid: "63d0b32305d3467ca589af857a661212" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 36549 } }
I20260808 11:37:30.866325 26573 ts_tablet_manager.cc:1428] T 8f7905fc4ba74a51a23d158937f688c1 P 149c0c323ccd4db383423eb5caa9b8c2: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260808 11:37:30.866811 26571 consensus_queue.cc:260] T 8f7905fc4ba74a51a23d158937f688c1 P 63d0b32305d3467ca589af857a661212 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d11b25dc5a734c0597109a22dfefc73f" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44351 } } peers { permanent_uuid: "149c0c323ccd4db383423eb5caa9b8c2" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41257 } } peers { permanent_uuid: "63d0b32305d3467ca589af857a661212" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 36549 } }
I20260808 11:37:30.871253 26571 ts_tablet_manager.cc:1428] T 8f7905fc4ba74a51a23d158937f688c1 P 63d0b32305d3467ca589af857a661212: Time spent starting tablet: real 0.017s user 0.011s sys 0.007s
I20260808 11:37:30.874836 26351 heartbeater.cc:503] Master 127.24.76.126:37591 was elected leader, sending a full tablet report...
I20260808 11:37:30.875846 26570 ts_tablet_manager.cc:1428] T 8f7905fc4ba74a51a23d158937f688c1 P d11b25dc5a734c0597109a22dfefc73f: Time spent starting tablet: real 0.025s user 0.017s sys 0.004s
W20260808 11:37:30.960775 26547 tablet.cc:2369] T 8f7905fc4ba74a51a23d158937f688c1 P 149c0c323ccd4db383423eb5caa9b8c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:37:30.985802 26352 tablet.cc:2369] T 8f7905fc4ba74a51a23d158937f688c1 P d11b25dc5a734c0597109a22dfefc73f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:37:31.000535 26130 tablet.cc:2369] T 8f7905fc4ba74a51a23d158937f688c1 P 63d0b32305d3467ca589af857a661212: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:37:31.024437 26578 raft_consensus.cc:482] T 8f7905fc4ba74a51a23d158937f688c1 P d11b25dc5a734c0597109a22dfefc73f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:37:31.024798 26578 raft_consensus.cc:504] T 8f7905fc4ba74a51a23d158937f688c1 P d11b25dc5a734c0597109a22dfefc73f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d11b25dc5a734c0597109a22dfefc73f" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44351 } } peers { permanent_uuid: "149c0c323ccd4db383423eb5caa9b8c2" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41257 } } peers { permanent_uuid: "63d0b32305d3467ca589af857a661212" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 36549 } }
I20260808 11:37:31.027218 26578 leader_election.cc:288] T 8f7905fc4ba74a51a23d158937f688c1 P d11b25dc5a734c0597109a22dfefc73f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 149c0c323ccd4db383423eb5caa9b8c2 (127.24.76.67:41257), 63d0b32305d3467ca589af857a661212 (127.24.76.65:36549)
I20260808 11:37:31.035597 26565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:31.027954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:35647 (local address 127.24.76.67:41257)
0808 11:37:31.028182 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:31.028189 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:31.028809 (+ 620us) server_negotiation.cc:407] Connection header received
0808 11:37:31.029074 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:31.029080 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:31.029160 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:31.029255 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:37:31.030302 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:31.031145 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:31.033388 (+ 2243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:31.034449 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:31.034571 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:31.034642 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:31.035134 (+ 492us) server_negotiation.cc:299] Negotiation successful
0808 11:37:31.035261 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 11:37:31.035617 26569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:31.029140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:51313 (local address 127.24.76.65:36549)
0808 11:37:31.029388 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:31.029394 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:31.029858 (+ 464us) server_negotiation.cc:407] Connection header received
0808 11:37:31.030045 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:31.030049 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:31.030106 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:31.030177 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:37:31.030785 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:31.031698 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:31.033650 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:31.034481 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:31.034675 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:31.034753 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:31.035234 (+ 481us) server_negotiation.cc:299] Negotiation successful
0808 11:37:31.035364 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 11:37:31.035665 26584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:31.027709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:41257 (local address 127.24.76.66:35647)
0808 11:37:31.028704 (+ 995us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:31.028723 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:31.028908 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:31.029477 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:31.029482 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:31.029518 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:37:31.030145 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:31.030158 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:31.031281 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:31.031286 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:31.033217 (+ 1931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:31.033230 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:31.034800 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:31.034808 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:31.034963 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:31.034999 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 11:37:31.035075 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 11:37:31.035163 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":814,"thread_start_us":505,"threads_started":1}
I20260808 11:37:31.035954 26585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:31.028852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:36549 (local address 127.24.76.66:51313)
0808 11:37:31.029750 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:31.029764 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:31.029911 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:31.030211 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:31.030216 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:31.030240 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:37:31.030635 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:31.030645 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:31.031852 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:31.031856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:31.033464 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:31.033480 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:31.034893 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:31.034899 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:31.035054 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:31.035088 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 11:37:31.035183 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 11:37:31.035302 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":793,"thread_start_us":590,"threads_started":1}
I20260808 11:37:31.036901 26485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f7905fc4ba74a51a23d158937f688c1" candidate_uuid: "d11b25dc5a734c0597109a22dfefc73f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "149c0c323ccd4db383423eb5caa9b8c2" is_pre_election: true
I20260808 11:37:31.036968 26058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f7905fc4ba74a51a23d158937f688c1" candidate_uuid: "d11b25dc5a734c0597109a22dfefc73f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63d0b32305d3467ca589af857a661212" is_pre_election: true
I20260808 11:37:31.037694 26485 raft_consensus.cc:2454] T 8f7905fc4ba74a51a23d158937f688c1 P 149c0c323ccd4db383423eb5caa9b8c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d11b25dc5a734c0597109a22dfefc73f in term 0.
I20260808 11:37:31.038559 26172 leader_election.cc:302] T 8f7905fc4ba74a51a23d158937f688c1 P d11b25dc5a734c0597109a22dfefc73f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 149c0c323ccd4db383423eb5caa9b8c2, d11b25dc5a734c0597109a22dfefc73f; no voters:
I20260808 11:37:31.038975 26578 raft_consensus.cc:2781] T 8f7905fc4ba74a51a23d158937f688c1 P d11b25dc5a734c0597109a22dfefc73f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:37:31.039187 26578 raft_consensus.cc:482] T 8f7905fc4ba74a51a23d158937f688c1 P d11b25dc5a734c0597109a22dfefc73f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:37:31.039371 26578 raft_consensus.cc:3037] T 8f7905fc4ba74a51a23d158937f688c1 P d11b25dc5a734c0597109a22dfefc73f [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:37:31.039597 26058 raft_consensus.cc:2454] T 8f7905fc4ba74a51a23d158937f688c1 P 63d0b32305d3467ca589af857a661212 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d11b25dc5a734c0597109a22dfefc73f in term 0.
I20260808 11:37:31.042152 26578 raft_consensus.cc:504] T 8f7905fc4ba74a51a23d158937f688c1 P d11b25dc5a734c0597109a22dfefc73f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d11b25dc5a734c0597109a22dfefc73f" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44351 } } peers { permanent_uuid: "149c0c323ccd4db383423eb5caa9b8c2" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41257 } } peers { permanent_uuid: "63d0b32305d3467ca589af857a661212" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 36549 } }
I20260808 11:37:31.042938 26578 leader_election.cc:288] T 8f7905fc4ba74a51a23d158937f688c1 P d11b25dc5a734c0597109a22dfefc73f [CANDIDATE]: Term 1 election: Requested vote from peers 149c0c323ccd4db383423eb5caa9b8c2 (127.24.76.67:41257), 63d0b32305d3467ca589af857a661212 (127.24.76.65:36549)
I20260808 11:37:31.043418 26058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f7905fc4ba74a51a23d158937f688c1" candidate_uuid: "d11b25dc5a734c0597109a22dfefc73f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63d0b32305d3467ca589af857a661212"
I20260808 11:37:31.043493 26485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f7905fc4ba74a51a23d158937f688c1" candidate_uuid: "d11b25dc5a734c0597109a22dfefc73f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "149c0c323ccd4db383423eb5caa9b8c2"
I20260808 11:37:31.043685 26058 raft_consensus.cc:3037] T 8f7905fc4ba74a51a23d158937f688c1 P 63d0b32305d3467ca589af857a661212 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:37:31.043705 26485 raft_consensus.cc:3037] T 8f7905fc4ba74a51a23d158937f688c1 P 149c0c323ccd4db383423eb5caa9b8c2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:37:31.047380 26058 raft_consensus.cc:2454] T 8f7905fc4ba74a51a23d158937f688c1 P 63d0b32305d3467ca589af857a661212 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d11b25dc5a734c0597109a22dfefc73f in term 1.
I20260808 11:37:31.047433 26485 raft_consensus.cc:2454] T 8f7905fc4ba74a51a23d158937f688c1 P 149c0c323ccd4db383423eb5caa9b8c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d11b25dc5a734c0597109a22dfefc73f in term 1.
I20260808 11:37:31.047784 26172 leader_election.cc:302] T 8f7905fc4ba74a51a23d158937f688c1 P d11b25dc5a734c0597109a22dfefc73f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63d0b32305d3467ca589af857a661212, d11b25dc5a734c0597109a22dfefc73f; no voters:
I20260808 11:37:31.048089 26578 raft_consensus.cc:2781] T 8f7905fc4ba74a51a23d158937f688c1 P d11b25dc5a734c0597109a22dfefc73f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:37:31.048626 26578 raft_consensus.cc:686] T 8f7905fc4ba74a51a23d158937f688c1 P d11b25dc5a734c0597109a22dfefc73f [term 1 LEADER]: Becoming Leader. State: Replica: d11b25dc5a734c0597109a22dfefc73f, State: Running, Role: LEADER
I20260808 11:37:31.049099 26578 consensus_queue.cc:237] T 8f7905fc4ba74a51a23d158937f688c1 P d11b25dc5a734c0597109a22dfefc73f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d11b25dc5a734c0597109a22dfefc73f" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44351 } } peers { permanent_uuid: "149c0c323ccd4db383423eb5caa9b8c2" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41257 } } peers { permanent_uuid: "63d0b32305d3467ca589af857a661212" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 36549 } }
I20260808 11:37:31.054854 25823 catalog_manager.cc:5495] T 8f7905fc4ba74a51a23d158937f688c1 P d11b25dc5a734c0597109a22dfefc73f reported cstate change: term changed from 0 to 1, leader changed from <none> to d11b25dc5a734c0597109a22dfefc73f (127.24.76.66). New cstate: current_term: 1 leader_uuid: "d11b25dc5a734c0597109a22dfefc73f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d11b25dc5a734c0597109a22dfefc73f" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "149c0c323ccd4db383423eb5caa9b8c2" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "63d0b32305d3467ca589af857a661212" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 36549 } health_report { overall_health: UNKNOWN } } }
Aug 08 11:37:31 dist-test-slave-tc2s krb5kdc[25756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.65@KRBTEST.COM
I20260808 11:37:31.104509 26569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:31.088154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37804 (local address 127.24.76.65:36549)
0808 11:37:31.088319 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:31.088324 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:31.089163 (+ 839us) server_negotiation.cc:407] Connection header received
0808 11:37:31.089331 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:31.089336 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:31.089399 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:31.089486 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:31.091446 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:31.092506 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:31.093743 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:31.094354 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:31.094453 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:31.098291 (+ 3838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:31.098307 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:31.098324 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:31.098360 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:31.102071 (+ 3711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:31.102619 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:31.102625 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:31.102627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:31.102759 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:31.103035 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:31.103038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:31.103040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:31.103462 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:31.103804 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:31.103941 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 11:37:31.104104 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 11:37:31 dist-test-slave-tc2s krb5kdc[25756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189051, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 11:37:31 dist-test-slave-tc2s krb5kdc[25756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189051, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.76.65@KRBTEST.COM
I20260808 11:37:31.154361 26569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:31.140245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37816 (local address 127.24.76.65:36549)
0808 11:37:31.140436 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:31.140442 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:31.140868 (+ 426us) server_negotiation.cc:407] Connection header received
0808 11:37:31.141059 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:31.141064 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:31.141129 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:31.141209 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:31.142774 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:31.143526 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:31.144731 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:31.145145 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:31.145250 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:31.149227 (+ 3977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:31.149232 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:31.149236 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:31.149269 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:31.152256 (+ 2987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:31.152802 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:31.152806 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:31.152808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:31.152951 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:31.153264 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:31.153267 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:31.153269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:31.153523 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:31.153711 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:31.153947 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 11:37:31.154118 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Aug 08 11:37:31 dist-test-slave-tc2s krb5kdc[25756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189051, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 11:37:31 dist-test-slave-tc2s krb5kdc[25756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189051, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.65@KRBTEST.COM
I20260808 11:37:31.204725 26569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:31.189061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37832 (local address 127.24.76.65:36549)
0808 11:37:31.189250 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:31.189258 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:31.189674 (+ 416us) server_negotiation.cc:407] Connection header received
0808 11:37:31.189936 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:31.189944 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:31.190018 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:31.190103 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:31.191793 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:31.192585 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:31.193885 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:31.194686 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:31.194800 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:31.199013 (+ 4213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:31.199020 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:31.199023 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:31.199056 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:31.201953 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:31.202643 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:31.202649 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:31.202652 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:31.202772 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:31.203116 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:31.203122 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:31.203125 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:31.203499 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:31.203768 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:31.204050 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 11:37:31.204272 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260808 11:37:31.207095 25989 tablet_service.cc:2908] Rejecting scan request for tablet 8f7905fc4ba74a51a23d158937f688c1: Uninitialized: clean time has not yet been initialized
I20260808 11:37:31.211154 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 25931
I20260808 11:37:31.223026 26569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:31.211895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37840 (local address 127.24.76.65:36549)
0808 11:37:31.212058 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:31.212063 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:31.212366 (+ 303us) server_negotiation.cc:407] Connection header received
0808 11:37:31.212481 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:31.212485 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:31.212548 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:31.212625 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:31.213672 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:31.214424 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:31.215515 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:31.215918 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:31.216017 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:31.217937 (+ 1920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:31.217945 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:31.217949 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:31.217981 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:31.220658 (+ 2677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:31.221267 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:31.221271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:31.221273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:31.221427 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:31.221695 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:31.221698 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:31.221700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:31.222083 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:31.222373 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:31.222519 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 11:37:31.222694 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 11:37:31.223661 26121 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:37:31.312417 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 25931
I20260808 11:37:31.337555 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 26139
Aug 08 11:37:31 dist-test-slave-tc2s krb5kdc[25756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189051, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.66@KRBTEST.COM
I20260808 11:37:31.351847 26615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:31.338309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47008 (local address 127.24.76.66:44351)
0808 11:37:31.338746 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:31.338751 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:31.338767 (+ 16us) server_negotiation.cc:407] Connection header received
0808 11:37:31.338826 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:31.338830 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:31.338890 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:31.338995 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:31.340112 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:31.340970 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:31.342277 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:31.342773 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:31.342866 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:31.346582 (+ 3716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:31.346597 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:31.346615 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:31.346647 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:31.349451 (+ 2804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:31.350229 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:31.350237 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:31.350239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:31.350371 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:31.350652 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:31.350655 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:31.350657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:31.351015 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:31.351235 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:31.351417 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 11:37:31.351585 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":224,"threads_started":1}
I20260808 11:37:31.352612 26341 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:37:31.443369 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 26139
I20260808 11:37:31.469300 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 26362
Aug 08 11:37:31 dist-test-slave-tc2s krb5kdc[25756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189051, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.67@KRBTEST.COM
I20260808 11:37:31.483507 26565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:31.470015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50482 (local address 127.24.76.67:41257)
0808 11:37:31.470171 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:31.470175 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:31.470219 (+ 44us) server_negotiation.cc:407] Connection header received
0808 11:37:31.470408 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:31.470411 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:31.470478 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:31.470562 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:31.471821 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:31.472554 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:31.473784 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:31.474285 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:31.474422 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:31.478277 (+ 3855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:31.478292 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:31.478320 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:31.478355 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:31.481118 (+ 2763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:31.481708 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:31.481715 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:31.481717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:31.481856 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:31.482276 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:31.482279 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:31.482281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:31.482659 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:31.482885 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:31.483091 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 11:37:31.483248 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 11:37:31.484078 26539 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:37:31.569837 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 26362
I20260808 11:37:31.592842 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 25778
I20260808 11:37:31.593989 25879 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:37:31.702077 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 25778
2026-08-08T11:37:31Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4312 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:37:31 dist-test-slave-tc2s krb5kdc[26625](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:37:31 dist-test-slave-tc2s krb5kdc[26625](info): set up 2 sockets
Aug 08 11:37:31 dist-test-slave-tc2s krb5kdc[26625](info): commencing operation
krb5kdc: starting...
W20260808 11:37:32.897090 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:37:33 dist-test-slave-tc2s krb5kdc[26625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:37:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:37:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
I20260808 11:37:33.078896 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:36339
--webserver_interface=127.24.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:42973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:36339
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:37:33.395709 26647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:37:33.395964 26647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:37:33.396097 26647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:37:33.403560 26647 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:37:33.403678 26647 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 11:37:33.403739 26647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:37:33.403826 26647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:37:33.403893 26647 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:37:33.403975 26647 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:37:33.416075 26647 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:42973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:36339
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:36339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:37:33.417207 26647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:37:33.418514 26647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:37:33.429867 26656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:33.429790 26660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:33.429843 26657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:37:33.430604 26647 server_base.cc:1034] running on GCE node
I20260808 11:37:33.431362 26647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:37:33.432617 26647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:37:33.433828 26647 hybrid_clock.cc:648] HybridClock initialized: now 1786189053433767 us; error 42 us; skew 500 ppm
I20260808 11:37:33.434141 26647 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:37:33 dist-test-slave-tc2s krb5kdc[26625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189053, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:37:33.436976 26647 init.cc:377] Logged in from keytab as kudu/127.24.76.126@KRBTEST.COM (short username kudu)
I20260808 11:37:33.438373 26647 webserver.cc:460] Webserver started at http://127.24.76.126:34201/ using document root <none> and password file <none>
I20260808 11:37:33.439026 26647 fs_manager.cc:359] Metadata directory not provided
I20260808 11:37:33.439127 26647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:37:33.439504 26647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:37:33.442400 26647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "974ca9bbdcd44562b076f7c2628d921c"
format_stamp: "Formatted at 2026-08-08 11:37:33 on dist-test-slave-tc2s"
server_key: "e406fc923bb76bc446ccd88d84caca20"
server_key_iv: "93d13c61b02222494e5e60a474c340cb"
server_key_version: "encryptionkey@0"
I20260808 11:37:33.443135 26647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "974ca9bbdcd44562b076f7c2628d921c"
format_stamp: "Formatted at 2026-08-08 11:37:33 on dist-test-slave-tc2s"
server_key: "e406fc923bb76bc446ccd88d84caca20"
server_key_iv: "93d13c61b02222494e5e60a474c340cb"
server_key_version: "encryptionkey@0"
I20260808 11:37:33.448267 26647 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260808 11:37:33.451689 26674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:33.452929 26647 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 11:37:33.453078 26647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "974ca9bbdcd44562b076f7c2628d921c"
format_stamp: "Formatted at 2026-08-08 11:37:33 on dist-test-slave-tc2s"
server_key: "e406fc923bb76bc446ccd88d84caca20"
server_key_iv: "93d13c61b02222494e5e60a474c340cb"
server_key_version: "encryptionkey@0"
I20260808 11:37:33.453266 26647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:37:33.470589 26647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:37:33.474167 26647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:37:33.474396 26647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:37:33.494422 26647 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:36339
I20260808 11:37:33.494464 26771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:36339 every 8 connection(s)
I20260808 11:37:33.495712 26647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:37:33.499315 26773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:37:33.504793 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 26647
I20260808 11:37:33.505044 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:37:33.505842 24881 external_mini_cluster.cc:1428] Setting key ce2cd6b8119d41ee6ce6f2a7aee0e00a
I20260808 11:37:33.510242 26773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 974ca9bbdcd44562b076f7c2628d921c: Bootstrap starting.
I20260808 11:37:33.513096 26773 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 974ca9bbdcd44562b076f7c2628d921c: Neither blocks nor log segments found. Creating new log.
I20260808 11:37:33.514088 26773 log.cc:824] T 00000000000000000000000000000000 P 974ca9bbdcd44562b076f7c2628d921c: Log is configured to *not* fsync() on all Append() calls
I20260808 11:37:33.516619 26773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 974ca9bbdcd44562b076f7c2628d921c: No bootstrap required, opened a new log
Aug 08 11:37:33 dist-test-slave-tc2s krb5kdc[26625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:37:33.525027 26773 raft_consensus.cc:348] T 00000000000000000000000000000000 P 974ca9bbdcd44562b076f7c2628d921c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "974ca9bbdcd44562b076f7c2628d921c" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 36339 } }
I20260808 11:37:33.525379 26773 raft_consensus.cc:374] T 00000000000000000000000000000000 P 974ca9bbdcd44562b076f7c2628d921c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:37:33.525511 26773 raft_consensus.cc:729] T 00000000000000000000000000000000 P 974ca9bbdcd44562b076f7c2628d921c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 974ca9bbdcd44562b076f7c2628d921c, State: Initialized, Role: FOLLOWER
I20260808 11:37:33.526175 26773 consensus_queue.cc:260] T 00000000000000000000000000000000 P 974ca9bbdcd44562b076f7c2628d921c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "974ca9bbdcd44562b076f7c2628d921c" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 36339 } }
I20260808 11:37:33.526443 26773 raft_consensus.cc:388] T 00000000000000000000000000000000 P 974ca9bbdcd44562b076f7c2628d921c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:37:33.526576 26773 raft_consensus.cc:482] T 00000000000000000000000000000000 P 974ca9bbdcd44562b076f7c2628d921c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:37:33.526741 26773 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 974ca9bbdcd44562b076f7c2628d921c [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:37:33.527479 26777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:33.507962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59522 (local address 127.24.76.126:36339)
0808 11:37:33.508731 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:33.508742 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:33.508779 (+ 37us) server_negotiation.cc:407] Connection header received
0808 11:37:33.510505 (+ 1726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:33.510529 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:33.510881 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:33.511203 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:33.512601 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:33.513941 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:33.515423 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:33.516066 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:33.516233 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:33.520519 (+ 4286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:33.520527 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:33.520541 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:33.520574 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:33.523489 (+ 2915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:33.524250 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:33.524256 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:33.524261 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:33.524414 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:33.524780 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:33.524783 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:33.524785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:33.525259 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:33.525543 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:33.525989 (+ 446us) server_negotiation.cc:299] Negotiation successful
0808 11:37:33.526315 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":349,"threads_started":1}
I20260808 11:37:33.529506 26773 raft_consensus.cc:504] T 00000000000000000000000000000000 P 974ca9bbdcd44562b076f7c2628d921c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "974ca9bbdcd44562b076f7c2628d921c" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 36339 } }
I20260808 11:37:33.530016 26773 leader_election.cc:302] T 00000000000000000000000000000000 P 974ca9bbdcd44562b076f7c2628d921c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 974ca9bbdcd44562b076f7c2628d921c; no voters:
I20260808 11:37:33.530561 26773 leader_election.cc:288] T 00000000000000000000000000000000 P 974ca9bbdcd44562b076f7c2628d921c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:37:33.530689 26780 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 974ca9bbdcd44562b076f7c2628d921c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:37:33.531309 26780 raft_consensus.cc:686] T 00000000000000000000000000000000 P 974ca9bbdcd44562b076f7c2628d921c [term 1 LEADER]: Becoming Leader. State: Replica: 974ca9bbdcd44562b076f7c2628d921c, State: Running, Role: LEADER
I20260808 11:37:33.531886 26780 consensus_queue.cc:237] T 00000000000000000000000000000000 P 974ca9bbdcd44562b076f7c2628d921c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "974ca9bbdcd44562b076f7c2628d921c" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 36339 } }
I20260808 11:37:33.532640 26773 sys_catalog.cc:564] T 00000000000000000000000000000000 P 974ca9bbdcd44562b076f7c2628d921c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:37:33.534893 26782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 974ca9bbdcd44562b076f7c2628d921c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 974ca9bbdcd44562b076f7c2628d921c. Latest consensus state: current_term: 1 leader_uuid: "974ca9bbdcd44562b076f7c2628d921c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "974ca9bbdcd44562b076f7c2628d921c" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 36339 } } }
I20260808 11:37:33.535292 26782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 974ca9bbdcd44562b076f7c2628d921c [sys.catalog]: This master's current role is: LEADER
I20260808 11:37:33.541016 26781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 974ca9bbdcd44562b076f7c2628d921c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "974ca9bbdcd44562b076f7c2628d921c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "974ca9bbdcd44562b076f7c2628d921c" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 36339 } } }
I20260808 11:37:33.541384 26781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 974ca9bbdcd44562b076f7c2628d921c [sys.catalog]: This master's current role is: LEADER
I20260808 11:37:33.544577 26791 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:37:33.549582 26791 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:37:33.557865 26791 catalog_manager.cc:1347] Generated new cluster ID: f16b2288d21846f9ad7e929d5d2388ef
I20260808 11:37:33.558022 26791 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:37:33.568112 26791 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:37:33.569272 26791 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:37:33.577508 26791 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 974ca9bbdcd44562b076f7c2628d921c: Generated new TSK 0
I20260808 11:37:33.578231 26791 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab.
Entry for principal kudu/127.24.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab.
I20260808 11:37:33.631644 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.65:0
--local_ip_for_outbound_sockets=127.24.76.65
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:36339
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:42973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:37:33.953245 26814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:37:33.953518 26814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:37:33.953656 26814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:37:33.960884 26814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:37:33.961000 26814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:37:33.961133 26814 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 11:37:33.961243 26814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.65
I20260808 11:37:33.970773 26814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:42973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.76.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.76.126:36339
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.65
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:37:33.971987 26814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:37:33.973238 26814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:37:33.985795 26825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:33.986143 26827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:33.986641 26829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:37:33.986667 26814 server_base.cc:1034] running on GCE node
I20260808 11:37:33.987264 26814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:37:33.988087 26814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:37:33.989300 26814 hybrid_clock.cc:648] HybridClock initialized: now 1786189053989238 us; error 41 us; skew 500 ppm
I20260808 11:37:33.989564 26814 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:37:33 dist-test-slave-tc2s krb5kdc[26625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189053, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:37:33.992540 26814 init.cc:377] Logged in from keytab as kudu/127.24.76.65@KRBTEST.COM (short username kudu)
I20260808 11:37:33.993865 26814 webserver.cc:460] Webserver started at http://127.24.76.65:40609/ using document root <none> and password file <none>
I20260808 11:37:33.994546 26814 fs_manager.cc:359] Metadata directory not provided
I20260808 11:37:33.994654 26814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:37:33.994946 26814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:37:33.997654 26814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-0/data/instance:
uuid: "bc1f5399d01846cba2ce3091796e003f"
format_stamp: "Formatted at 2026-08-08 11:37:33 on dist-test-slave-tc2s"
server_key: "767ac47590ed2c8382e9b8000a943ecf"
server_key_iv: "ccd67e9a44a35d3ecae452ad2cd19804"
server_key_version: "encryptionkey@0"
I20260808 11:37:33.998426 26814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance:
uuid: "bc1f5399d01846cba2ce3091796e003f"
format_stamp: "Formatted at 2026-08-08 11:37:33 on dist-test-slave-tc2s"
server_key: "767ac47590ed2c8382e9b8000a943ecf"
server_key_iv: "ccd67e9a44a35d3ecae452ad2cd19804"
server_key_version: "encryptionkey@0"
I20260808 11:37:34.003356 26814 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 11:37:34.006716 26839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:34.007912 26814 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:37:34.008069 26814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-0/wal
uuid: "bc1f5399d01846cba2ce3091796e003f"
format_stamp: "Formatted at 2026-08-08 11:37:33 on dist-test-slave-tc2s"
server_key: "767ac47590ed2c8382e9b8000a943ecf"
server_key_iv: "ccd67e9a44a35d3ecae452ad2cd19804"
server_key_version: "encryptionkey@0"
I20260808 11:37:34.008275 26814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:37:34.018839 26814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:37:34.022358 26814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:37:34.022586 26814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:37:34.025043 26814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:37:34.025125 26814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:34.025334 26814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:37:34.025434 26814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 11:37:34 dist-test-slave-tc2s krb5kdc[26625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189053, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:37:34.047607 26777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:34.025623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:40751 (local address 127.24.76.126:36339)
0808 11:37:34.025816 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:34.025823 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:34.027062 (+ 1239us) server_negotiation.cc:407] Connection header received
0808 11:37:34.029040 (+ 1978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:34.029047 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:34.029130 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:34.029215 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:34.031232 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:34.032182 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:34.034769 (+ 2587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:34.035431 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:34.035589 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:34.040303 (+ 4714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:34.040312 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:34.040318 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:34.040362 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:34.043852 (+ 3490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:34.044782 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:34.044788 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:34.044791 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:34.044959 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:34.045521 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:34.045527 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:34.045531 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:34.045888 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:34.046063 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:34.046999 (+ 936us) server_negotiation.cc:299] Negotiation successful
0808 11:37:34.047238 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 11:37:34.049181 26851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:34.026079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36339 (local address 127.24.76.65:40751)
0808 11:37:34.026890 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:34.026936 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:34.028777 (+ 1841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:34.029461 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:34.029469 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:34.030193 (+ 724us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:37:34.030992 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:34.031021 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:34.032344 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:34.032351 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:34.034561 (+ 2210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:34.034577 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:34.035711 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:34.035720 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:34.035911 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:34.036661 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:37:34.036697 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:37:34.040009 (+ 3312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:37:34.043984 (+ 3975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:34.043992 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:34.044007 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:34.044640 (+ 633us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:34.045082 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:34.045087 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:34.045091 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:34.045377 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:34.046054 (+ 677us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:37:34.046062 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:37:34.046538 (+ 476us) client_negotiation.cc:769] Sending connection context
0808 11:37:34.046881 (+ 343us) client_negotiation.cc:240] Negotiation successful
0808 11:37:34.047454 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":353,"threads_started":1}
I20260808 11:37:34.067723 26814 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.65:43749
I20260808 11:37:34.067760 27033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.65:43749 every 8 connection(s)
I20260808 11:37:34.069049 26814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
I20260808 11:37:34.069375 26777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:34.056651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:32903 (local address 127.24.76.126:36339)
0808 11:37:34.056825 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:34.056831 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:34.057552 (+ 721us) server_negotiation.cc:407] Connection header received
0808 11:37:34.057796 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:34.057802 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:34.057883 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:34.058013 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:34.058856 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:34.059900 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:34.061938 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:34.062456 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:34.062572 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:34.064218 (+ 1646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:34.064224 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:34.064229 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:34.064266 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:34.066725 (+ 2459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:34.067425 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:34.067430 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:34.067432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:34.067545 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:34.067937 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:34.067942 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:34.067946 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:34.068209 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:34.068352 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:34.068810 (+ 458us) server_negotiation.cc:299] Negotiation successful
0808 11:37:34.069025 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 11:37:34.069473 26992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:34.056697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36339 (local address 127.24.76.65:32903)
0808 11:37:34.057449 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:34.057469 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:34.057646 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:34.058047 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:34.058053 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:34.058461 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:37:34.058737 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:34.058747 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:34.060066 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:34.060075 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:34.061782 (+ 1707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:34.061797 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:34.062660 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:34.062668 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:34.062819 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:34.063366 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:37:34.063382 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:37:34.064036 (+ 654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:37:34.066843 (+ 2807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:34.066848 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:34.066852 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:34.067308 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:34.067667 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:34.067672 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:34.067675 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:34.067802 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:34.068337 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:37:34.068342 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:37:34.068579 (+ 237us) client_negotiation.cc:769] Sending connection context
0808 11:37:34.068703 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 11:37:34.069026 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":451,"threads_started":1}
I20260808 11:37:34.071349 27034 heartbeater.cc:347] Connected to a master server at 127.24.76.126:36339
I20260808 11:37:34.071575 27034 heartbeater.cc:464] Registering TS with master...
I20260808 11:37:34.072315 27034 heartbeater.cc:511] Master 127.24.76.126:36339 requested a full tablet report, sending...
I20260808 11:37:34.074311 26707 ts_manager.cc:194] Registered new tserver with Master: bc1f5399d01846cba2ce3091796e003f (127.24.76.65:43749)
I20260808 11:37:34.075656 26707 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.65@KRBTEST.COM'} at 127.24.76.65:40751
I20260808 11:37:34.078912 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 26814
I20260808 11:37:34.079113 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance
I20260808 11:37:34.079733 24881 external_mini_cluster.cc:1428] Setting key 5c50ee5fbac706a9a8c3922a20be14e5
WARNING: no policy specified for kudu/127.24.76.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab.
Entry for principal kudu/127.24.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab.
I20260808 11:37:34.121290 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.66:0
--local_ip_for_outbound_sockets=127.24.76.66
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:36339
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:42973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:37:34.437216 27038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:37:34.437472 27038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:37:34.437605 27038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:37:34.445032 27038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:37:34.445138 27038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:37:34.445261 27038 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 11:37:34.445362 27038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.66
I20260808 11:37:34.454905 27038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:42973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.76.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.76.126:36339
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.66
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:37:34.456067 27038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:37:34.457444 27038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:37:34.470219 27044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:34.470176 27045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:34.470259 27049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:37:34.471038 27038 server_base.cc:1034] running on GCE node
I20260808 11:37:34.471614 27038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:37:34.472445 27038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:37:34.473670 27038 hybrid_clock.cc:648] HybridClock initialized: now 1786189054473601 us; error 46 us; skew 500 ppm
I20260808 11:37:34.473996 27038 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:37:34 dist-test-slave-tc2s krb5kdc[26625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189054, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:37:34.477062 27038 init.cc:377] Logged in from keytab as kudu/127.24.76.66@KRBTEST.COM (short username kudu)
I20260808 11:37:34.478533 27038 webserver.cc:460] Webserver started at http://127.24.76.66:38135/ using document root <none> and password file <none>
I20260808 11:37:34.479175 27038 fs_manager.cc:359] Metadata directory not provided
I20260808 11:37:34.479274 27038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:37:34.479609 27038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:37:34.482455 27038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-1/data/instance:
uuid: "d0863aa6a60e4e748cf8bab0a94a953b"
format_stamp: "Formatted at 2026-08-08 11:37:34 on dist-test-slave-tc2s"
server_key: "38b6d0cf17f9047e3afa3ac36d695361"
server_key_iv: "8e5352004f82e3db88167ec0f58ab6dc"
server_key_version: "encryptionkey@0"
I20260808 11:37:34.483201 27038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance:
uuid: "d0863aa6a60e4e748cf8bab0a94a953b"
format_stamp: "Formatted at 2026-08-08 11:37:34 on dist-test-slave-tc2s"
server_key: "38b6d0cf17f9047e3afa3ac36d695361"
server_key_iv: "8e5352004f82e3db88167ec0f58ab6dc"
server_key_version: "encryptionkey@0"
I20260808 11:37:34.487951 27038 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260808 11:37:34.491334 27059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:34.492551 27038 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:37:34.492688 27038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-1/wal
uuid: "d0863aa6a60e4e748cf8bab0a94a953b"
format_stamp: "Formatted at 2026-08-08 11:37:34 on dist-test-slave-tc2s"
server_key: "38b6d0cf17f9047e3afa3ac36d695361"
server_key_iv: "8e5352004f82e3db88167ec0f58ab6dc"
server_key_version: "encryptionkey@0"
I20260808 11:37:34.492879 27038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:37:34.524021 27038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:37:34.527705 27038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:37:34.527956 27038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:37:34.530539 27038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:37:34.530619 27038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:34.530777 27038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:37:34.530893 27038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Aug 08 11:37:34 dist-test-slave-tc2s krb5kdc[26625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189054, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:37:34.553102 26777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:34.531016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:44831 (local address 127.24.76.126:36339)
0808 11:37:34.531206 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:34.531212 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:34.532603 (+ 1391us) server_negotiation.cc:407] Connection header received
0808 11:37:34.535318 (+ 2715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:34.535324 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:34.535417 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:34.535529 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:34.537865 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:34.538797 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:34.541764 (+ 2967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:34.542442 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:34.542577 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:34.546475 (+ 3898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:34.546480 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:34.546490 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:34.546528 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:34.549848 (+ 3320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:34.550603 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:34.550609 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:34.550613 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:34.550769 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:34.551251 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:34.551256 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:34.551260 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:34.551664 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:34.551806 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:34.552480 (+ 674us) server_negotiation.cc:299] Negotiation successful
0808 11:37:34.552740 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 11:37:34.553918 27070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:34.531595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36339 (local address 127.24.76.66:44831)
0808 11:37:34.532428 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:34.532490 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:34.535010 (+ 2520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:34.535839 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:34.535853 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:34.536672 (+ 819us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:37:34.537657 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:34.537688 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:34.538980 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:34.538986 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:34.541563 (+ 2577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:34.541583 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:34.542703 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:34.542711 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:34.542936 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:34.543524 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:37:34.543545 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:37:34.546179 (+ 2634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:37:34.550007 (+ 3828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:34.550014 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:34.550024 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:34.550474 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:34.550912 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:34.550917 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:34.550926 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:34.551149 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:34.551794 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:37:34.551800 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:37:34.552091 (+ 291us) client_negotiation.cc:769] Sending connection context
0808 11:37:34.552353 (+ 262us) client_negotiation.cc:240] Negotiation successful
0808 11:37:34.552785 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":331,"threads_started":1}
I20260808 11:37:34.574280 26777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:34.559443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:56509 (local address 127.24.76.126:36339)
0808 11:37:34.559629 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:34.559636 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:34.560171 (+ 535us) server_negotiation.cc:407] Connection header received
0808 11:37:34.560303 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:34.560309 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:34.560387 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:34.560488 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:34.561294 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:34.562429 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:34.564596 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:34.565111 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:34.565232 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:34.567740 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:34.567745 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:34.567748 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:34.567776 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:34.570620 (+ 2844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:34.571486 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:34.571492 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:34.571495 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:34.571616 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:34.571958 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:34.571963 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:34.571967 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:34.572373 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:34.572510 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:34.573779 (+ 1269us) server_negotiation.cc:299] Negotiation successful
0808 11:37:34.574050 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 11:37:34.574373 27168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:34.559443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36339 (local address 127.24.76.66:56509)
0808 11:37:34.560086 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:34.560104 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:34.560209 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:34.560498 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:34.560501 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:34.560891 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:37:34.561182 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:34.561191 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:34.562601 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:34.562606 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:34.564455 (+ 1849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:34.564471 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:34.565477 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:34.565483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:34.565639 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:34.566333 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:37:34.566354 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:37:34.567523 (+ 1169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:37:34.570742 (+ 3219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:34.570746 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:34.570750 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:34.571380 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:34.571725 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:34.571730 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:34.571734 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:34.571859 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:34.572567 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:37:34.572571 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:37:34.572779 (+ 208us) client_negotiation.cc:769] Sending connection context
0808 11:37:34.572955 (+ 176us) client_negotiation.cc:240] Negotiation successful
0808 11:37:34.573263 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":355,"threads_started":1}
I20260808 11:37:34.582412 27038 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.66:44735
I20260808 11:37:34.582459 27251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.66:44735 every 8 connection(s)
I20260808 11:37:34.583585 27038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
I20260808 11:37:34.585433 27253 heartbeater.cc:347] Connected to a master server at 127.24.76.126:36339
I20260808 11:37:34.585649 27253 heartbeater.cc:464] Registering TS with master...
I20260808 11:37:34.586294 27253 heartbeater.cc:511] Master 127.24.76.126:36339 requested a full tablet report, sending...
I20260808 11:37:34.586968 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 27038
I20260808 11:37:34.587180 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance
I20260808 11:37:34.587682 24881 external_mini_cluster.cc:1428] Setting key 129cfae53dd32e5410d010e94743794b
I20260808 11:37:34.587728 26707 ts_manager.cc:194] Registered new tserver with Master: d0863aa6a60e4e748cf8bab0a94a953b (127.24.76.66:44735)
I20260808 11:37:34.588630 26707 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.66@KRBTEST.COM'} at 127.24.76.66:44831
WARNING: no policy specified for kudu/127.24.76.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab.
Entry for principal kudu/127.24.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab.
I20260808 11:37:34.630074 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.67:0
--local_ip_for_outbound_sockets=127.24.76.67
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:36339
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:42973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:37:34.943099 27257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:37:34.943436 27257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:37:34.943575 27257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:37:34.950558 27257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:37:34.950682 27257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:37:34.950779 27257 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 11:37:34.950881 27257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.67
I20260808 11:37:34.960247 27257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:42973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.76.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.76.126:36339
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.67
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:37:34.961381 27257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:37:34.962635 27257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:37:34.974905 27263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:34.974885 27264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:34.974949 27266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:37:34.975853 27257 server_base.cc:1034] running on GCE node
I20260808 11:37:34.976416 27257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:37:34.977244 27257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:37:34.978463 27257 hybrid_clock.cc:648] HybridClock initialized: now 1786189054978391 us; error 42 us; skew 500 ppm
I20260808 11:37:34.978747 27257 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:37:34 dist-test-slave-tc2s krb5kdc[26625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189054, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:37:34.981812 27257 init.cc:377] Logged in from keytab as kudu/127.24.76.67@KRBTEST.COM (short username kudu)
I20260808 11:37:34.983207 27257 webserver.cc:460] Webserver started at http://127.24.76.67:40407/ using document root <none> and password file <none>
I20260808 11:37:34.983852 27257 fs_manager.cc:359] Metadata directory not provided
I20260808 11:37:34.983960 27257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:37:34.984264 27257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:37:34.987058 27257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-2/data/instance:
uuid: "750df1e92cbb4b5198dc9205b5cc125d"
format_stamp: "Formatted at 2026-08-08 11:37:34 on dist-test-slave-tc2s"
server_key: "de77f35bfcbb0f629acd0acdf165db20"
server_key_iv: "b2be7904e6db37b8013a47a1dd442d5b"
server_key_version: "encryptionkey@0"
I20260808 11:37:34.987879 27257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance:
uuid: "750df1e92cbb4b5198dc9205b5cc125d"
format_stamp: "Formatted at 2026-08-08 11:37:34 on dist-test-slave-tc2s"
server_key: "de77f35bfcbb0f629acd0acdf165db20"
server_key_iv: "b2be7904e6db37b8013a47a1dd442d5b"
server_key_version: "encryptionkey@0"
I20260808 11:37:34.992636 27257 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 11:37:34.995855 27274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:34.997190 27257 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 11:37:34.997325 27257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-2/wal
uuid: "750df1e92cbb4b5198dc9205b5cc125d"
format_stamp: "Formatted at 2026-08-08 11:37:34 on dist-test-slave-tc2s"
server_key: "de77f35bfcbb0f629acd0acdf165db20"
server_key_iv: "b2be7904e6db37b8013a47a1dd442d5b"
server_key_version: "encryptionkey@0"
I20260808 11:37:34.997516 27257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:37:35.031155 27257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:37:35.035319 27257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:37:35.035569 27257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:37:35.038050 27257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:37:35.038134 27257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:35.038273 27257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:37:35.038331 27257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 11:37:35 dist-test-slave-tc2s krb5kdc[26625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189054, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:37:35.060768 26777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:35.038661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:44865 (local address 127.24.76.126:36339)
0808 11:37:35.038859 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:35.038865 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:35.040180 (+ 1315us) server_negotiation.cc:407] Connection header received
0808 11:37:35.042352 (+ 2172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:35.042358 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:35.042443 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:35.042531 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:35.044698 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.045864 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.048111 (+ 2247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.048839 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.048975 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:35.053322 (+ 4347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:35.053328 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:35.053334 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:35.053376 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:35.057118 (+ 3742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:35.058060 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:35.058065 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:35.058069 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:35.058241 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:35.058773 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:35.058778 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:35.058782 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:35.059127 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:35.059270 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:35.060197 (+ 927us) server_negotiation.cc:299] Negotiation successful
0808 11:37:35.060440 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 11:37:35.061738 27296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:35.039175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36339 (local address 127.24.76.67:44865)
0808 11:37:35.040004 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:35.040052 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:35.042066 (+ 2014us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:35.042831 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:35.042842 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:35.043591 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:37:35.044481 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:35.044524 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.046065 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.046074 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:35.047950 (+ 1876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:35.047963 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.049046 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.049051 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:35.049226 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:35.049861 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:37:35.049882 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:37:35.053038 (+ 3156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:37:35.057266 (+ 4228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:35.057276 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:35.057291 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:35.057874 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:35.058351 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:35.058360 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:35.058363 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:35.058660 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:35.059333 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:37:35.059341 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:37:35.059810 (+ 469us) client_negotiation.cc:769] Sending connection context
0808 11:37:35.060065 (+ 255us) client_negotiation.cc:240] Negotiation successful
0808 11:37:35.060584 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":354,"threads_started":1}
I20260808 11:37:35.078168 27034 heartbeater.cc:503] Master 127.24.76.126:36339 was elected leader, sending a full tablet report...
I20260808 11:37:35.082479 26777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:35.070056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:44439 (local address 127.24.76.126:36339)
0808 11:37:35.070211 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:35.070217 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:35.070235 (+ 18us) server_negotiation.cc:407] Connection header received
0808 11:37:35.070286 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:35.070290 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:35.070374 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:35.070464 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:35.071413 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.072101 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.073370 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.073810 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.073942 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:35.076248 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:35.076253 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:35.076256 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:35.076285 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:35.078938 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:35.080258 (+ 1320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:35.080264 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:35.080267 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:35.080444 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:35.080868 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:35.080871 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:35.080873 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:35.081148 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:35.081352 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:35.082079 (+ 727us) server_negotiation.cc:299] Negotiation successful
0808 11:37:35.082231 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 11:37:35.082662 27403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:35.067231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36339 (local address 127.24.76.67:44439)
0808 11:37:35.067840 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:35.067865 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:35.067971 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:35.070515 (+ 2544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:35.070521 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:35.070933 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:37:35.071228 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:35.071237 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.072239 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.072244 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:35.073240 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:35.073251 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.074047 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.074053 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:35.074201 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:35.074909 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:37:35.074935 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:37:35.076077 (+ 1142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:37:35.079703 (+ 3626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:35.079707 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:35.079709 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:35.080147 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:35.080584 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:35.080590 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:35.080594 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:35.080765 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:35.081326 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:37:35.081331 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:37:35.081589 (+ 258us) client_negotiation.cc:769] Sending connection context
0808 11:37:35.081784 (+ 195us) client_negotiation.cc:240] Negotiation successful
0808 11:37:35.082171 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":312,"threads_started":1}
I20260808 11:37:35.092878 27257 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.67:37105
I20260808 11:37:35.092911 27482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.67:37105 every 8 connection(s)
I20260808 11:37:35.094004 27257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
I20260808 11:37:35.095530 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 27257
I20260808 11:37:35.095736 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance
I20260808 11:37:35.095846 27483 heartbeater.cc:347] Connected to a master server at 127.24.76.126:36339
I20260808 11:37:35.096078 27483 heartbeater.cc:464] Registering TS with master...
I20260808 11:37:35.096161 24881 external_mini_cluster.cc:1428] Setting key f45dd971d6912548b0e720e7db4ff10a
I20260808 11:37:35.096697 27483 heartbeater.cc:511] Master 127.24.76.126:36339 requested a full tablet report, sending...
I20260808 11:37:35.098227 26706 ts_manager.cc:194] Registered new tserver with Master: 750df1e92cbb4b5198dc9205b5cc125d (127.24.76.67:37105)
I20260808 11:37:35.099273 24881 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:37:35.099373 26706 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.67@KRBTEST.COM'} at 127.24.76.67:44865
Aug 08 11:37:35 dist-test-slave-tc2s krb5kdc[26625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189055, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 11:37:35 dist-test-slave-tc2s krb5kdc[26625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189055, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:37:35.147475 26777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:35.134651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59526 (local address 127.24.76.126:36339)
0808 11:37:35.134811 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:35.134816 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:35.135158 (+ 342us) server_negotiation.cc:407] Connection header received
0808 11:37:35.135309 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:35.135313 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:35.135377 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:35.135474 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:35.136783 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.137534 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.138773 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.139167 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.139261 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:35.143065 (+ 3804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:35.143070 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:35.143074 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:35.143109 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:35.145398 (+ 2289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:35.145996 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:35.146000 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:35.146002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:35.146128 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:35.146432 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:35.146435 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:35.146437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:35.146684 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:35.146867 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:35.147074 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 11:37:35.147233 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 11:37:35.151371 26706 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59526:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:37:35.158277 26706 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:37:35.193765 27506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:35.169742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53886 (local address 127.24.76.65:43749)
0808 11:37:35.186244 (+ 16502us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:35.186249 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:35.186273 (+ 24us) server_negotiation.cc:407] Connection header received
0808 11:37:35.186353 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:35.186359 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:35.186543 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:35.186723 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:37:35.187509 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.189014 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.191753 (+ 2739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.192704 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.192827 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:35.193055 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:35.193320 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 11:37:35.193479 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16370,"thread_start_us":17146,"threads_started":1}
I20260808 11:37:35.193862 27504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:35.169565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:43749 (local address 127.0.0.1:53886)
0808 11:37:35.182947 (+ 13382us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:35.182990 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:35.183176 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:35.186770 (+ 3594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:35.186777 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:35.186807 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:37:35.187284 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:35.187298 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.189144 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.189150 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:35.191584 (+ 2434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:35.191603 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.192916 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.192922 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:35.193110 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:35.193146 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 11:37:35.193264 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 11:37:35.193440 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13112,"mutex_wait_us":42,"thread_start_us":12964,"threads_started":1}
I20260808 11:37:35.194728 27510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:35.169733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:44735 (local address 127.0.0.1:48902)
0808 11:37:35.186899 (+ 17166us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:35.186915 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:35.187055 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:35.187886 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:35.187891 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:35.187916 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:37:35.188297 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:35.188309 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.189767 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.189771 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:35.191619 (+ 1848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:35.191631 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.193660 (+ 2029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.193666 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:35.193834 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:35.194115 (+ 281us) client_negotiation.cc:769] Sending connection context
0808 11:37:35.194204 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 11:37:35.194326 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17033,"mutex_wait_us":33,"thread_start_us":17960,"threads_started":1}
I20260808 11:37:35.194855 27509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:35.170110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48902 (local address 127.24.76.66:44735)
0808 11:37:35.182352 (+ 12242us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:35.182363 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:35.187127 (+ 4764us) server_negotiation.cc:407] Connection header received
0808 11:37:35.187256 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:35.187263 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:35.187476 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:35.187670 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:37:35.188508 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.189635 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.191804 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.193210 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.193332 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:35.193511 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:35.194406 (+ 895us) server_negotiation.cc:299] Negotiation successful
0808 11:37:35.194584 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12041,"thread_start_us":13241,"threads_started":1}
I20260808 11:37:35.196280 27507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:35.171739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:37105 (local address 127.0.0.1:60018)
0808 11:37:35.187546 (+ 15807us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:35.187563 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:35.187748 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:35.188267 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:35.188271 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:35.188297 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:37:35.188641 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:35.188651 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.190562 (+ 1911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.190567 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:35.192917 (+ 2350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:35.192931 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.195560 (+ 2629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.195567 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:35.195687 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:35.195709 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 11:37:35.195786 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 11:37:35.195901 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15688,"mutex_wait_us":2,"thread_start_us":14897,"threads_started":1}
I20260808 11:37:35.196959 27511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:35.172238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60018 (local address 127.24.76.67:37105)
0808 11:37:35.186451 (+ 14213us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:35.186460 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:35.187651 (+ 1191us) server_negotiation.cc:407] Connection header received
0808 11:37:35.187943 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:35.187950 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:35.188099 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:35.188246 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:37:35.188819 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.190386 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.193104 (+ 2718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.194494 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.194610 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:35.195418 (+ 808us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:35.196446 (+ 1028us) server_negotiation.cc:299] Negotiation successful
0808 11:37:35.196603 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14039,"thread_start_us":13864,"threads_started":1}
I20260808 11:37:35.198256 27144 tablet_service.cc:1467] Processing CreateTablet for tablet 4c50e3e1714b45ea8027bfe884ae422c (DEFAULT_TABLE table=test-table [id=70abc5d118b945908c06b707a13ddb6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:37:35.198266 26921 tablet_service.cc:1467] Processing CreateTablet for tablet 4c50e3e1714b45ea8027bfe884ae422c (DEFAULT_TABLE table=test-table [id=70abc5d118b945908c06b707a13ddb6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:37:35.199571 27144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c50e3e1714b45ea8027bfe884ae422c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:37:35.200073 26921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c50e3e1714b45ea8027bfe884ae422c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:37:35.200361 27370 tablet_service.cc:1467] Processing CreateTablet for tablet 4c50e3e1714b45ea8027bfe884ae422c (DEFAULT_TABLE table=test-table [id=70abc5d118b945908c06b707a13ddb6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:37:35.201704 27370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c50e3e1714b45ea8027bfe884ae422c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:37:35.210505 27512 tablet_bootstrap.cc:492] T 4c50e3e1714b45ea8027bfe884ae422c P d0863aa6a60e4e748cf8bab0a94a953b: Bootstrap starting.
I20260808 11:37:35.213641 27512 tablet_bootstrap.cc:654] T 4c50e3e1714b45ea8027bfe884ae422c P d0863aa6a60e4e748cf8bab0a94a953b: Neither blocks nor log segments found. Creating new log.
I20260808 11:37:35.214564 27513 tablet_bootstrap.cc:492] T 4c50e3e1714b45ea8027bfe884ae422c P bc1f5399d01846cba2ce3091796e003f: Bootstrap starting.
I20260808 11:37:35.215082 27512 log.cc:824] T 4c50e3e1714b45ea8027bfe884ae422c P d0863aa6a60e4e748cf8bab0a94a953b: Log is configured to *not* fsync() on all Append() calls
I20260808 11:37:35.216305 27514 tablet_bootstrap.cc:492] T 4c50e3e1714b45ea8027bfe884ae422c P 750df1e92cbb4b5198dc9205b5cc125d: Bootstrap starting.
I20260808 11:37:35.218492 27513 tablet_bootstrap.cc:654] T 4c50e3e1714b45ea8027bfe884ae422c P bc1f5399d01846cba2ce3091796e003f: Neither blocks nor log segments found. Creating new log.
I20260808 11:37:35.219692 27512 tablet_bootstrap.cc:492] T 4c50e3e1714b45ea8027bfe884ae422c P d0863aa6a60e4e748cf8bab0a94a953b: No bootstrap required, opened a new log
I20260808 11:37:35.219846 27513 log.cc:824] T 4c50e3e1714b45ea8027bfe884ae422c P bc1f5399d01846cba2ce3091796e003f: Log is configured to *not* fsync() on all Append() calls
I20260808 11:37:35.219969 27512 ts_tablet_manager.cc:1397] T 4c50e3e1714b45ea8027bfe884ae422c P d0863aa6a60e4e748cf8bab0a94a953b: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
I20260808 11:37:35.220011 27514 tablet_bootstrap.cc:654] T 4c50e3e1714b45ea8027bfe884ae422c P 750df1e92cbb4b5198dc9205b5cc125d: Neither blocks nor log segments found. Creating new log.
I20260808 11:37:35.221598 27514 log.cc:824] T 4c50e3e1714b45ea8027bfe884ae422c P 750df1e92cbb4b5198dc9205b5cc125d: Log is configured to *not* fsync() on all Append() calls
I20260808 11:37:35.223749 27513 tablet_bootstrap.cc:492] T 4c50e3e1714b45ea8027bfe884ae422c P bc1f5399d01846cba2ce3091796e003f: No bootstrap required, opened a new log
I20260808 11:37:35.224020 27513 ts_tablet_manager.cc:1397] T 4c50e3e1714b45ea8027bfe884ae422c P bc1f5399d01846cba2ce3091796e003f: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260808 11:37:35.224526 27514 tablet_bootstrap.cc:492] T 4c50e3e1714b45ea8027bfe884ae422c P 750df1e92cbb4b5198dc9205b5cc125d: No bootstrap required, opened a new log
I20260808 11:37:35.224822 27514 ts_tablet_manager.cc:1397] T 4c50e3e1714b45ea8027bfe884ae422c P 750df1e92cbb4b5198dc9205b5cc125d: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260808 11:37:35.231802 27512 raft_consensus.cc:348] T 4c50e3e1714b45ea8027bfe884ae422c P d0863aa6a60e4e748cf8bab0a94a953b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } }
I20260808 11:37:35.232365 27512 raft_consensus.cc:374] T 4c50e3e1714b45ea8027bfe884ae422c P d0863aa6a60e4e748cf8bab0a94a953b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:37:35.232503 27512 raft_consensus.cc:729] T 4c50e3e1714b45ea8027bfe884ae422c P d0863aa6a60e4e748cf8bab0a94a953b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0863aa6a60e4e748cf8bab0a94a953b, State: Initialized, Role: FOLLOWER
I20260808 11:37:35.233248 27512 consensus_queue.cc:260] T 4c50e3e1714b45ea8027bfe884ae422c P d0863aa6a60e4e748cf8bab0a94a953b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } }
I20260808 11:37:35.234712 27253 heartbeater.cc:503] Master 127.24.76.126:36339 was elected leader, sending a full tablet report...
I20260808 11:37:35.235088 27514 raft_consensus.cc:348] T 4c50e3e1714b45ea8027bfe884ae422c P 750df1e92cbb4b5198dc9205b5cc125d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } }
I20260808 11:37:35.235387 27514 raft_consensus.cc:374] T 4c50e3e1714b45ea8027bfe884ae422c P 750df1e92cbb4b5198dc9205b5cc125d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:37:35.235445 27512 ts_tablet_manager.cc:1428] T 4c50e3e1714b45ea8027bfe884ae422c P d0863aa6a60e4e748cf8bab0a94a953b: Time spent starting tablet: real 0.015s user 0.011s sys 0.005s
I20260808 11:37:35.235532 27514 raft_consensus.cc:729] T 4c50e3e1714b45ea8027bfe884ae422c P 750df1e92cbb4b5198dc9205b5cc125d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 750df1e92cbb4b5198dc9205b5cc125d, State: Initialized, Role: FOLLOWER
I20260808 11:37:35.236255 27514 consensus_queue.cc:260] T 4c50e3e1714b45ea8027bfe884ae422c P 750df1e92cbb4b5198dc9205b5cc125d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } }
I20260808 11:37:35.237756 27483 heartbeater.cc:503] Master 127.24.76.126:36339 was elected leader, sending a full tablet report...
I20260808 11:37:35.237525 27513 raft_consensus.cc:348] T 4c50e3e1714b45ea8027bfe884ae422c P bc1f5399d01846cba2ce3091796e003f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } }
I20260808 11:37:35.238046 27513 raft_consensus.cc:374] T 4c50e3e1714b45ea8027bfe884ae422c P bc1f5399d01846cba2ce3091796e003f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:37:35.238212 27513 raft_consensus.cc:729] T 4c50e3e1714b45ea8027bfe884ae422c P bc1f5399d01846cba2ce3091796e003f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc1f5399d01846cba2ce3091796e003f, State: Initialized, Role: FOLLOWER
I20260808 11:37:35.238857 27513 consensus_queue.cc:260] T 4c50e3e1714b45ea8027bfe884ae422c P bc1f5399d01846cba2ce3091796e003f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } }
I20260808 11:37:35.239933 27514 ts_tablet_manager.cc:1428] T 4c50e3e1714b45ea8027bfe884ae422c P 750df1e92cbb4b5198dc9205b5cc125d: Time spent starting tablet: real 0.015s user 0.005s sys 0.009s
I20260808 11:37:35.240797 27513 ts_tablet_manager.cc:1428] T 4c50e3e1714b45ea8027bfe884ae422c P bc1f5399d01846cba2ce3091796e003f: Time spent starting tablet: real 0.017s user 0.012s sys 0.005s
W20260808 11:37:35.321853 27035 tablet.cc:2369] T 4c50e3e1714b45ea8027bfe884ae422c P bc1f5399d01846cba2ce3091796e003f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:37:35.335538 27254 tablet.cc:2369] T 4c50e3e1714b45ea8027bfe884ae422c P d0863aa6a60e4e748cf8bab0a94a953b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:37:35.345479 27484 tablet.cc:2369] T 4c50e3e1714b45ea8027bfe884ae422c P 750df1e92cbb4b5198dc9205b5cc125d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:37:35.558918 27523 raft_consensus.cc:482] T 4c50e3e1714b45ea8027bfe884ae422c P bc1f5399d01846cba2ce3091796e003f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:37:35.559273 27523 raft_consensus.cc:504] T 4c50e3e1714b45ea8027bfe884ae422c P bc1f5399d01846cba2ce3091796e003f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } }
I20260808 11:37:35.560974 27523 leader_election.cc:288] T 4c50e3e1714b45ea8027bfe884ae422c P bc1f5399d01846cba2ce3091796e003f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0863aa6a60e4e748cf8bab0a94a953b (127.24.76.66:44735), 750df1e92cbb4b5198dc9205b5cc125d (127.24.76.67:37105)
I20260808 11:37:35.565100 27520 raft_consensus.cc:482] T 4c50e3e1714b45ea8027bfe884ae422c P 750df1e92cbb4b5198dc9205b5cc125d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:37:35.565464 27520 raft_consensus.cc:504] T 4c50e3e1714b45ea8027bfe884ae422c P 750df1e92cbb4b5198dc9205b5cc125d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } }
I20260808 11:37:35.567095 27520 leader_election.cc:288] T 4c50e3e1714b45ea8027bfe884ae422c P 750df1e92cbb4b5198dc9205b5cc125d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc1f5399d01846cba2ce3091796e003f (127.24.76.65:43749), d0863aa6a60e4e748cf8bab0a94a953b (127.24.76.66:44735)
I20260808 11:37:35.569912 27509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:35.561856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:50439 (local address 127.24.76.66:44735)
0808 11:37:35.562105 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:35.562111 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:35.562461 (+ 350us) server_negotiation.cc:407] Connection header received
0808 11:37:35.562719 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:35.562723 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:35.562786 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:35.562862 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:37:35.563684 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.564520 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.567307 (+ 2787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.568684 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.568828 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:35.568903 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:35.569497 (+ 594us) server_negotiation.cc:299] Negotiation successful
0808 11:37:35.569630 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260808 11:37:35.570576 27511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:35.561513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:40431 (local address 127.24.76.67:37105)
0808 11:37:35.561944 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:35.561958 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:35.562666 (+ 708us) server_negotiation.cc:407] Connection header received
0808 11:37:35.562830 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:35.562834 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:35.562893 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:35.562972 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:37:35.563718 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.564400 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.567384 (+ 2984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.569444 (+ 2060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.569582 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:35.569645 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:35.570190 (+ 545us) server_negotiation.cc:299] Negotiation successful
0808 11:37:35.570321 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256}
I20260808 11:37:35.571179 27524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:35.561319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:44735 (local address 127.24.76.65:50439)
0808 11:37:35.562340 (+ 1021us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:35.562364 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:35.562572 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:35.562894 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:35.562898 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:35.562923 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:37:35.563511 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:35.563524 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.564665 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.564672 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:35.567104 (+ 2432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:35.567119 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.569120 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.569127 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:35.569300 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:35.569333 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 11:37:35.570607 (+ 1274us) client_negotiation.cc:240] Negotiation successful
0808 11:37:35.570737 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"mutex_wait_us":2,"thread_start_us":431,"threads_started":1}
I20260808 11:37:35.571201 27526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:35.561421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:37105 (local address 127.24.76.65:40431)
0808 11:37:35.562568 (+ 1147us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:35.562583 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:35.562705 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:35.563032 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:35.563036 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:35.563077 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:37:35.563528 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:35.563539 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.564542 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.564547 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:35.567228 (+ 2681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:35.567242 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.569825 (+ 2583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.569830 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:35.570006 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:35.570039 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 11:37:35.570713 (+ 674us) client_negotiation.cc:240] Negotiation successful
0808 11:37:35.570814 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1051,"mutex_wait_us":38,"thread_start_us":854,"threads_started":1}
I20260808 11:37:35.572322 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c50e3e1714b45ea8027bfe884ae422c" candidate_uuid: "bc1f5399d01846cba2ce3091796e003f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" is_pre_election: true
I20260808 11:37:35.572825 27181 raft_consensus.cc:2454] T 4c50e3e1714b45ea8027bfe884ae422c P d0863aa6a60e4e748cf8bab0a94a953b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc1f5399d01846cba2ce3091796e003f in term 0.
I20260808 11:37:35.573472 26842 leader_election.cc:302] T 4c50e3e1714b45ea8027bfe884ae422c P bc1f5399d01846cba2ce3091796e003f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc1f5399d01846cba2ce3091796e003f, d0863aa6a60e4e748cf8bab0a94a953b; no voters:
I20260808 11:37:35.573886 27523 raft_consensus.cc:2781] T 4c50e3e1714b45ea8027bfe884ae422c P bc1f5399d01846cba2ce3091796e003f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:37:35.574139 27523 raft_consensus.cc:482] T 4c50e3e1714b45ea8027bfe884ae422c P bc1f5399d01846cba2ce3091796e003f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:37:35.574251 27523 raft_consensus.cc:3037] T 4c50e3e1714b45ea8027bfe884ae422c P bc1f5399d01846cba2ce3091796e003f [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:37:35.575429 27404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c50e3e1714b45ea8027bfe884ae422c" candidate_uuid: "bc1f5399d01846cba2ce3091796e003f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "750df1e92cbb4b5198dc9205b5cc125d" is_pre_election: true
I20260808 11:37:35.576059 27404 raft_consensus.cc:2454] T 4c50e3e1714b45ea8027bfe884ae422c P 750df1e92cbb4b5198dc9205b5cc125d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc1f5399d01846cba2ce3091796e003f in term 0.
I20260808 11:37:35.577821 27523 raft_consensus.cc:504] T 4c50e3e1714b45ea8027bfe884ae422c P bc1f5399d01846cba2ce3091796e003f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } }
I20260808 11:37:35.578943 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c50e3e1714b45ea8027bfe884ae422c" candidate_uuid: "bc1f5399d01846cba2ce3091796e003f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0863aa6a60e4e748cf8bab0a94a953b"
I20260808 11:37:35.579150 27181 raft_consensus.cc:3037] T 4c50e3e1714b45ea8027bfe884ae422c P d0863aa6a60e4e748cf8bab0a94a953b [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:37:35.579555 27523 leader_election.cc:288] T 4c50e3e1714b45ea8027bfe884ae422c P bc1f5399d01846cba2ce3091796e003f [CANDIDATE]: Term 1 election: Requested vote from peers d0863aa6a60e4e748cf8bab0a94a953b (127.24.76.66:44735), 750df1e92cbb4b5198dc9205b5cc125d (127.24.76.67:37105)
I20260808 11:37:35.580051 27404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c50e3e1714b45ea8027bfe884ae422c" candidate_uuid: "bc1f5399d01846cba2ce3091796e003f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "750df1e92cbb4b5198dc9205b5cc125d"
I20260808 11:37:35.580255 27404 raft_consensus.cc:3037] T 4c50e3e1714b45ea8027bfe884ae422c P 750df1e92cbb4b5198dc9205b5cc125d [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:37:35.581856 27506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:35.568916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:51301 (local address 127.24.76.65:43749)
0808 11:37:35.569132 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:35.569137 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:35.574575 (+ 5438us) server_negotiation.cc:407] Connection header received
0808 11:37:35.574858 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:35.574864 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:35.574928 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:35.574998 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:37:35.576062 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.577219 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.579452 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.580769 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.580889 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:35.580964 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:35.581501 (+ 537us) server_negotiation.cc:299] Negotiation successful
0808 11:37:35.581621 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 11:37:35.582638 27529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:35.567151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:43749 (local address 127.24.76.67:51301)
0808 11:37:35.574480 (+ 7329us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:35.574500 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:35.574688 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:35.575184 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:35.575188 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:35.575213 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:37:35.575786 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:35.575803 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.577358 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.577361 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:35.579233 (+ 1872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:35.579249 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.581141 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.581147 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:35.581320 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:35.581353 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 11:37:35.582168 (+ 815us) client_negotiation.cc:240] Negotiation successful
0808 11:37:35.582303 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7159,"thread_start_us":7395,"threads_started":1}
I20260808 11:37:35.583961 27404 raft_consensus.cc:2454] T 4c50e3e1714b45ea8027bfe884ae422c P 750df1e92cbb4b5198dc9205b5cc125d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc1f5399d01846cba2ce3091796e003f in term 1.
I20260808 11:37:35.583938 26955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c50e3e1714b45ea8027bfe884ae422c" candidate_uuid: "750df1e92cbb4b5198dc9205b5cc125d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc1f5399d01846cba2ce3091796e003f" is_pre_election: true
I20260808 11:37:35.584347 26844 leader_election.cc:302] T 4c50e3e1714b45ea8027bfe884ae422c P bc1f5399d01846cba2ce3091796e003f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 750df1e92cbb4b5198dc9205b5cc125d, bc1f5399d01846cba2ce3091796e003f; no voters:
I20260808 11:37:35.584618 27181 raft_consensus.cc:2454] T 4c50e3e1714b45ea8027bfe884ae422c P d0863aa6a60e4e748cf8bab0a94a953b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc1f5399d01846cba2ce3091796e003f in term 1.
I20260808 11:37:35.584692 26955 raft_consensus.cc:2379] T 4c50e3e1714b45ea8027bfe884ae422c P bc1f5399d01846cba2ce3091796e003f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 750df1e92cbb4b5198dc9205b5cc125d in current term 1: Already voted for candidate bc1f5399d01846cba2ce3091796e003f in this term.
I20260808 11:37:35.585436 27523 raft_consensus.cc:2781] T 4c50e3e1714b45ea8027bfe884ae422c P bc1f5399d01846cba2ce3091796e003f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:37:35.585875 27509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:35.568020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:38127 (local address 127.24.76.66:44735)
0808 11:37:35.570558 (+ 2538us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:35.570568 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:35.574915 (+ 4347us) server_negotiation.cc:407] Connection header received
0808 11:37:35.575175 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:35.575180 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:35.575251 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:35.575309 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:37:35.576747 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.578002 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.580871 (+ 2869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.582760 (+ 1889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.583287 (+ 527us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:35.583351 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:35.585395 (+ 2044us) server_negotiation.cc:299] Negotiation successful
0808 11:37:35.585547 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2094,"thread_start_us":3643,"threads_started":1}
I20260808 11:37:35.586611 27527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:35.567471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:44735 (local address 127.24.76.67:38127)
0808 11:37:35.574828 (+ 7357us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:35.574845 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:35.575013 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:35.575496 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:35.575504 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:35.575528 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:37:35.576543 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:35.576559 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.578168 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.578173 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:35.580685 (+ 2512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:35.580701 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.585018 (+ 4317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.585024 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:35.585201 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:35.585236 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 11:37:35.586134 (+ 898us) client_negotiation.cc:240] Negotiation successful
0808 11:37:35.586284 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7233,"thread_start_us":6602,"threads_started":1}
I20260808 11:37:35.587034 27523 raft_consensus.cc:686] T 4c50e3e1714b45ea8027bfe884ae422c P bc1f5399d01846cba2ce3091796e003f [term 1 LEADER]: Becoming Leader. State: Replica: bc1f5399d01846cba2ce3091796e003f, State: Running, Role: LEADER
I20260808 11:37:35.587618 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c50e3e1714b45ea8027bfe884ae422c" candidate_uuid: "750df1e92cbb4b5198dc9205b5cc125d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" is_pre_election: true
I20260808 11:37:35.587538 27523 consensus_queue.cc:237] T 4c50e3e1714b45ea8027bfe884ae422c P bc1f5399d01846cba2ce3091796e003f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } }
I20260808 11:37:35.587917 27181 raft_consensus.cc:2379] T 4c50e3e1714b45ea8027bfe884ae422c P d0863aa6a60e4e748cf8bab0a94a953b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 750df1e92cbb4b5198dc9205b5cc125d in current term 1: Already voted for candidate bc1f5399d01846cba2ce3091796e003f in this term.
I20260808 11:37:35.588589 27277 leader_election.cc:302] T 4c50e3e1714b45ea8027bfe884ae422c P 750df1e92cbb4b5198dc9205b5cc125d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 750df1e92cbb4b5198dc9205b5cc125d; no voters: bc1f5399d01846cba2ce3091796e003f, d0863aa6a60e4e748cf8bab0a94a953b
I20260808 11:37:35.588989 27520 raft_consensus.cc:2726] T 4c50e3e1714b45ea8027bfe884ae422c P 750df1e92cbb4b5198dc9205b5cc125d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 11:37:35.592432 26706 catalog_manager.cc:5495] T 4c50e3e1714b45ea8027bfe884ae422c P bc1f5399d01846cba2ce3091796e003f reported cstate change: term changed from 0 to 1, leader changed from <none> to bc1f5399d01846cba2ce3091796e003f (127.24.76.65). New cstate: current_term: 1 leader_uuid: "bc1f5399d01846cba2ce3091796e003f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } health_report { overall_health: UNKNOWN } } }
I20260808 11:37:35.657953 27506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:35.651133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53890 (local address 127.24.76.65:43749)
0808 11:37:35.651346 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:35.651353 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:35.651829 (+ 476us) server_negotiation.cc:407] Connection header received
0808 11:37:35.652152 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:35.652156 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:35.652235 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:35.652347 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:37:35.652914 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.653681 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.655456 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:35.655964 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:35.656072 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:35.656623 (+ 551us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:37:35.657183 (+ 560us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:37:35.657333 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:35.657377 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 11:37:35.657466 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 11:37:35.669200 27404 raft_consensus.cc:1261] T 4c50e3e1714b45ea8027bfe884ae422c P 750df1e92cbb4b5198dc9205b5cc125d [term 1 FOLLOWER]: Refusing update from remote peer bc1f5399d01846cba2ce3091796e003f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:37:35.670243 27531 consensus_queue.cc:1035] T 4c50e3e1714b45ea8027bfe884ae422c P bc1f5399d01846cba2ce3091796e003f [LEADER]: Connected to new peer: Peer: permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:37:35.670665 27181 raft_consensus.cc:1261] T 4c50e3e1714b45ea8027bfe884ae422c P d0863aa6a60e4e748cf8bab0a94a953b [term 1 FOLLOWER]: Refusing update from remote peer bc1f5399d01846cba2ce3091796e003f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:37:35.671476 27531 consensus_queue.cc:1035] T 4c50e3e1714b45ea8027bfe884ae422c P bc1f5399d01846cba2ce3091796e003f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:37:35.683079 27537 mvcc.cc:205] Tried to move back new op lower bound from 7316230372009086976 to 7316230371691200512. Current Snapshot: MvccSnapshot[applied={T|T < 7316230372009086976}]
I20260808 11:37:35.704826 26706 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59526:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:37:35.705117 26706 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59526:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:37:35.708992 26706 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 974ca9bbdcd44562b076f7c2628d921c: Sending DeleteTablet for 3 replicas of tablet 4c50e3e1714b45ea8027bfe884ae422c
I20260808 11:37:35.710296 26921 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c50e3e1714b45ea8027bfe884ae422c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:37:35 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:53886
I20260808 11:37:35.710453 27370 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c50e3e1714b45ea8027bfe884ae422c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:37:35 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:60018
I20260808 11:37:35.710896 27144 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c50e3e1714b45ea8027bfe884ae422c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:37:35 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:48902
I20260808 11:37:35.717526 26706 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59526:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:37:35.719045 26706 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:37:35.725754 27554 tablet_replica.cc:330] T 4c50e3e1714b45ea8027bfe884ae422c P d0863aa6a60e4e748cf8bab0a94a953b: stopping tablet replica
I20260808 11:37:35.727130 27551 tablet_replica.cc:330] T 4c50e3e1714b45ea8027bfe884ae422c P 750df1e92cbb4b5198dc9205b5cc125d: stopping tablet replica
I20260808 11:37:35.727679 27554 raft_consensus.cc:2229] T 4c50e3e1714b45ea8027bfe884ae422c P d0863aa6a60e4e748cf8bab0a94a953b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:37:35.727684 27551 raft_consensus.cc:2229] T 4c50e3e1714b45ea8027bfe884ae422c P 750df1e92cbb4b5198dc9205b5cc125d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:37:35.728212 27554 raft_consensus.cc:2258] T 4c50e3e1714b45ea8027bfe884ae422c P d0863aa6a60e4e748cf8bab0a94a953b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:37:35.728384 27551 raft_consensus.cc:2258] T 4c50e3e1714b45ea8027bfe884ae422c P 750df1e92cbb4b5198dc9205b5cc125d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:37:35.728808 27553 tablet_replica.cc:330] T 4c50e3e1714b45ea8027bfe884ae422c P bc1f5399d01846cba2ce3091796e003f: stopping tablet replica
I20260808 11:37:35.729177 27553 raft_consensus.cc:2229] T 4c50e3e1714b45ea8027bfe884ae422c P bc1f5399d01846cba2ce3091796e003f [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:37:35.729242 26921 tablet_service.cc:1467] Processing CreateTablet for tablet e8a548497c4a47feadc568fc4fa549d7 (DEFAULT_TABLE table=test-table [id=3fe3e7334d024caaa0fe4a5ded36112e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:37:35.729914 26921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8a548497c4a47feadc568fc4fa549d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:37:35.729984 27553 raft_consensus.cc:2258] T 4c50e3e1714b45ea8027bfe884ae422c P bc1f5399d01846cba2ce3091796e003f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:37:35.730847 27370 tablet_service.cc:1467] Processing CreateTablet for tablet e8a548497c4a47feadc568fc4fa549d7 (DEFAULT_TABLE table=test-table [id=3fe3e7334d024caaa0fe4a5ded36112e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:37:35.731009 27144 tablet_service.cc:1467] Processing CreateTablet for tablet e8a548497c4a47feadc568fc4fa549d7 (DEFAULT_TABLE table=test-table [id=3fe3e7334d024caaa0fe4a5ded36112e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:37:35.731281 27370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8a548497c4a47feadc568fc4fa549d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:37:35.731437 27144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8a548497c4a47feadc568fc4fa549d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:37:35.731930 27554 ts_tablet_manager.cc:1905] T 4c50e3e1714b45ea8027bfe884ae422c P d0863aa6a60e4e748cf8bab0a94a953b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:37:35.733958 27551 ts_tablet_manager.cc:1905] T 4c50e3e1714b45ea8027bfe884ae422c P 750df1e92cbb4b5198dc9205b5cc125d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:37:35.735282 27553 ts_tablet_manager.cc:1905] T 4c50e3e1714b45ea8027bfe884ae422c P bc1f5399d01846cba2ce3091796e003f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:37:35.741882 27551 ts_tablet_manager.cc:1918] T 4c50e3e1714b45ea8027bfe884ae422c P 750df1e92cbb4b5198dc9205b5cc125d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:37:35.742147 27551 log.cc:1196] T 4c50e3e1714b45ea8027bfe884ae422c P 750df1e92cbb4b5198dc9205b5cc125d: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-2/wal/wals/4c50e3e1714b45ea8027bfe884ae422c
I20260808 11:37:35.742838 27551 ts_tablet_manager.cc:1939] T 4c50e3e1714b45ea8027bfe884ae422c P 750df1e92cbb4b5198dc9205b5cc125d: Deleting consensus metadata
I20260808 11:37:35.744913 27555 tablet_bootstrap.cc:492] T e8a548497c4a47feadc568fc4fa549d7 P d0863aa6a60e4e748cf8bab0a94a953b: Bootstrap starting.
I20260808 11:37:35.745217 26679 catalog_manager.cc:4841] TS 750df1e92cbb4b5198dc9205b5cc125d (127.24.76.67:37105): tablet 4c50e3e1714b45ea8027bfe884ae422c (table test-table [id=70abc5d118b945908c06b707a13ddb6e]) successfully deleted
I20260808 11:37:35.747380 27555 tablet_bootstrap.cc:654] T e8a548497c4a47feadc568fc4fa549d7 P d0863aa6a60e4e748cf8bab0a94a953b: Neither blocks nor log segments found. Creating new log.
I20260808 11:37:35.749284 27554 ts_tablet_manager.cc:1918] T 4c50e3e1714b45ea8027bfe884ae422c P d0863aa6a60e4e748cf8bab0a94a953b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:37:35.749578 27554 log.cc:1196] T 4c50e3e1714b45ea8027bfe884ae422c P d0863aa6a60e4e748cf8bab0a94a953b: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-1/wal/wals/4c50e3e1714b45ea8027bfe884ae422c
I20260808 11:37:35.750311 27554 ts_tablet_manager.cc:1939] T 4c50e3e1714b45ea8027bfe884ae422c P d0863aa6a60e4e748cf8bab0a94a953b: Deleting consensus metadata
I20260808 11:37:35.753436 26676 catalog_manager.cc:4841] TS d0863aa6a60e4e748cf8bab0a94a953b (127.24.76.66:44735): tablet 4c50e3e1714b45ea8027bfe884ae422c (table test-table [id=70abc5d118b945908c06b707a13ddb6e]) successfully deleted
I20260808 11:37:35.754783 27557 tablet_bootstrap.cc:492] T e8a548497c4a47feadc568fc4fa549d7 P bc1f5399d01846cba2ce3091796e003f: Bootstrap starting.
I20260808 11:37:35.759091 27557 tablet_bootstrap.cc:654] T e8a548497c4a47feadc568fc4fa549d7 P bc1f5399d01846cba2ce3091796e003f: Neither blocks nor log segments found. Creating new log.
I20260808 11:37:35.761806 27556 tablet_bootstrap.cc:492] T e8a548497c4a47feadc568fc4fa549d7 P 750df1e92cbb4b5198dc9205b5cc125d: Bootstrap starting.
I20260808 11:37:35.764205 27555 tablet_bootstrap.cc:492] T e8a548497c4a47feadc568fc4fa549d7 P d0863aa6a60e4e748cf8bab0a94a953b: No bootstrap required, opened a new log
I20260808 11:37:35.764359 27555 ts_tablet_manager.cc:1397] T e8a548497c4a47feadc568fc4fa549d7 P d0863aa6a60e4e748cf8bab0a94a953b: Time spent bootstrapping tablet: real 0.020s user 0.011s sys 0.007s
I20260808 11:37:35.765534 27556 tablet_bootstrap.cc:654] T e8a548497c4a47feadc568fc4fa549d7 P 750df1e92cbb4b5198dc9205b5cc125d: Neither blocks nor log segments found. Creating new log.
I20260808 11:37:35.765659 27555 raft_consensus.cc:348] T e8a548497c4a47feadc568fc4fa549d7 P d0863aa6a60e4e748cf8bab0a94a953b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } }
I20260808 11:37:35.766089 27555 raft_consensus.cc:374] T e8a548497c4a47feadc568fc4fa549d7 P d0863aa6a60e4e748cf8bab0a94a953b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:37:35.766244 27555 raft_consensus.cc:729] T e8a548497c4a47feadc568fc4fa549d7 P d0863aa6a60e4e748cf8bab0a94a953b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0863aa6a60e4e748cf8bab0a94a953b, State: Initialized, Role: FOLLOWER
I20260808 11:37:35.766577 27555 consensus_queue.cc:260] T e8a548497c4a47feadc568fc4fa549d7 P d0863aa6a60e4e748cf8bab0a94a953b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } }
I20260808 11:37:35.767483 27555 ts_tablet_manager.cc:1428] T e8a548497c4a47feadc568fc4fa549d7 P d0863aa6a60e4e748cf8bab0a94a953b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 11:37:35.766793 27553 ts_tablet_manager.cc:1918] T 4c50e3e1714b45ea8027bfe884ae422c P bc1f5399d01846cba2ce3091796e003f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:37:35.768205 27553 log.cc:1196] T 4c50e3e1714b45ea8027bfe884ae422c P bc1f5399d01846cba2ce3091796e003f: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-0/wal/wals/4c50e3e1714b45ea8027bfe884ae422c
I20260808 11:37:35.768563 27557 tablet_bootstrap.cc:492] T e8a548497c4a47feadc568fc4fa549d7 P bc1f5399d01846cba2ce3091796e003f: No bootstrap required, opened a new log
I20260808 11:37:35.768766 27557 ts_tablet_manager.cc:1397] T e8a548497c4a47feadc568fc4fa549d7 P bc1f5399d01846cba2ce3091796e003f: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.006s
I20260808 11:37:35.768949 27553 ts_tablet_manager.cc:1939] T 4c50e3e1714b45ea8027bfe884ae422c P bc1f5399d01846cba2ce3091796e003f: Deleting consensus metadata
I20260808 11:37:35.771121 26677 catalog_manager.cc:4841] TS bc1f5399d01846cba2ce3091796e003f (127.24.76.65:43749): tablet 4c50e3e1714b45ea8027bfe884ae422c (table test-table [id=70abc5d118b945908c06b707a13ddb6e]) successfully deleted
I20260808 11:37:35.771872 27556 tablet_bootstrap.cc:492] T e8a548497c4a47feadc568fc4fa549d7 P 750df1e92cbb4b5198dc9205b5cc125d: No bootstrap required, opened a new log
I20260808 11:37:35.771785 27557 raft_consensus.cc:348] T e8a548497c4a47feadc568fc4fa549d7 P bc1f5399d01846cba2ce3091796e003f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } }
I20260808 11:37:35.772053 27556 ts_tablet_manager.cc:1397] T e8a548497c4a47feadc568fc4fa549d7 P 750df1e92cbb4b5198dc9205b5cc125d: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.003s
I20260808 11:37:35.772117 27557 raft_consensus.cc:374] T e8a548497c4a47feadc568fc4fa549d7 P bc1f5399d01846cba2ce3091796e003f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:37:35.772240 27557 raft_consensus.cc:729] T e8a548497c4a47feadc568fc4fa549d7 P bc1f5399d01846cba2ce3091796e003f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc1f5399d01846cba2ce3091796e003f, State: Initialized, Role: FOLLOWER
I20260808 11:37:35.772537 27557 consensus_queue.cc:260] T e8a548497c4a47feadc568fc4fa549d7 P bc1f5399d01846cba2ce3091796e003f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } }
I20260808 11:37:35.773255 27556 raft_consensus.cc:348] T e8a548497c4a47feadc568fc4fa549d7 P 750df1e92cbb4b5198dc9205b5cc125d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } }
I20260808 11:37:35.773578 27556 raft_consensus.cc:374] T e8a548497c4a47feadc568fc4fa549d7 P 750df1e92cbb4b5198dc9205b5cc125d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:37:35.773759 27557 ts_tablet_manager.cc:1428] T e8a548497c4a47feadc568fc4fa549d7 P bc1f5399d01846cba2ce3091796e003f: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20260808 11:37:35.773778 27556 raft_consensus.cc:729] T e8a548497c4a47feadc568fc4fa549d7 P 750df1e92cbb4b5198dc9205b5cc125d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 750df1e92cbb4b5198dc9205b5cc125d, State: Initialized, Role: FOLLOWER
I20260808 11:37:35.774228 27556 consensus_queue.cc:260] T e8a548497c4a47feadc568fc4fa549d7 P 750df1e92cbb4b5198dc9205b5cc125d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } }
I20260808 11:37:35.774852 27556 ts_tablet_manager.cc:1428] T e8a548497c4a47feadc568fc4fa549d7 P 750df1e92cbb4b5198dc9205b5cc125d: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260808 11:37:36.077412 27523 raft_consensus.cc:482] T e8a548497c4a47feadc568fc4fa549d7 P bc1f5399d01846cba2ce3091796e003f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:37:36.077612 27523 raft_consensus.cc:504] T e8a548497c4a47feadc568fc4fa549d7 P bc1f5399d01846cba2ce3091796e003f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } }
I20260808 11:37:36.078339 27523 leader_election.cc:288] T e8a548497c4a47feadc568fc4fa549d7 P bc1f5399d01846cba2ce3091796e003f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0863aa6a60e4e748cf8bab0a94a953b (127.24.76.66:44735), 750df1e92cbb4b5198dc9205b5cc125d (127.24.76.67:37105)
I20260808 11:37:36.078962 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8a548497c4a47feadc568fc4fa549d7" candidate_uuid: "bc1f5399d01846cba2ce3091796e003f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" is_pre_election: true
I20260808 11:37:36.079149 27402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8a548497c4a47feadc568fc4fa549d7" candidate_uuid: "bc1f5399d01846cba2ce3091796e003f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "750df1e92cbb4b5198dc9205b5cc125d" is_pre_election: true
I20260808 11:37:36.079259 27181 raft_consensus.cc:2454] T e8a548497c4a47feadc568fc4fa549d7 P d0863aa6a60e4e748cf8bab0a94a953b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc1f5399d01846cba2ce3091796e003f in term 0.
I20260808 11:37:36.079417 27402 raft_consensus.cc:2454] T e8a548497c4a47feadc568fc4fa549d7 P 750df1e92cbb4b5198dc9205b5cc125d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc1f5399d01846cba2ce3091796e003f in term 0.
I20260808 11:37:36.079779 26842 leader_election.cc:302] T e8a548497c4a47feadc568fc4fa549d7 P bc1f5399d01846cba2ce3091796e003f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc1f5399d01846cba2ce3091796e003f, d0863aa6a60e4e748cf8bab0a94a953b; no voters:
I20260808 11:37:36.080142 27523 raft_consensus.cc:2781] T e8a548497c4a47feadc568fc4fa549d7 P bc1f5399d01846cba2ce3091796e003f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:37:36.080247 27523 raft_consensus.cc:482] T e8a548497c4a47feadc568fc4fa549d7 P bc1f5399d01846cba2ce3091796e003f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:37:36.080397 27523 raft_consensus.cc:3037] T e8a548497c4a47feadc568fc4fa549d7 P bc1f5399d01846cba2ce3091796e003f [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:37:36.082872 27523 raft_consensus.cc:504] T e8a548497c4a47feadc568fc4fa549d7 P bc1f5399d01846cba2ce3091796e003f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } }
I20260808 11:37:36.083878 27523 leader_election.cc:288] T e8a548497c4a47feadc568fc4fa549d7 P bc1f5399d01846cba2ce3091796e003f [CANDIDATE]: Term 1 election: Requested vote from peers d0863aa6a60e4e748cf8bab0a94a953b (127.24.76.66:44735), 750df1e92cbb4b5198dc9205b5cc125d (127.24.76.67:37105)
I20260808 11:37:36.083935 27181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8a548497c4a47feadc568fc4fa549d7" candidate_uuid: "bc1f5399d01846cba2ce3091796e003f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0863aa6a60e4e748cf8bab0a94a953b"
I20260808 11:37:36.084124 27181 raft_consensus.cc:3037] T e8a548497c4a47feadc568fc4fa549d7 P d0863aa6a60e4e748cf8bab0a94a953b [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:37:36.084276 27402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8a548497c4a47feadc568fc4fa549d7" candidate_uuid: "bc1f5399d01846cba2ce3091796e003f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "750df1e92cbb4b5198dc9205b5cc125d"
I20260808 11:37:36.084511 27402 raft_consensus.cc:3037] T e8a548497c4a47feadc568fc4fa549d7 P 750df1e92cbb4b5198dc9205b5cc125d [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:37:36.087483 27181 raft_consensus.cc:2454] T e8a548497c4a47feadc568fc4fa549d7 P d0863aa6a60e4e748cf8bab0a94a953b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc1f5399d01846cba2ce3091796e003f in term 1.
I20260808 11:37:36.088074 26842 leader_election.cc:302] T e8a548497c4a47feadc568fc4fa549d7 P bc1f5399d01846cba2ce3091796e003f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc1f5399d01846cba2ce3091796e003f, d0863aa6a60e4e748cf8bab0a94a953b; no voters:
I20260808 11:37:36.088299 27402 raft_consensus.cc:2454] T e8a548497c4a47feadc568fc4fa549d7 P 750df1e92cbb4b5198dc9205b5cc125d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc1f5399d01846cba2ce3091796e003f in term 1.
I20260808 11:37:36.088415 27523 raft_consensus.cc:2781] T e8a548497c4a47feadc568fc4fa549d7 P bc1f5399d01846cba2ce3091796e003f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:37:36.089004 27523 raft_consensus.cc:686] T e8a548497c4a47feadc568fc4fa549d7 P bc1f5399d01846cba2ce3091796e003f [term 1 LEADER]: Becoming Leader. State: Replica: bc1f5399d01846cba2ce3091796e003f, State: Running, Role: LEADER
I20260808 11:37:36.089277 27523 consensus_queue.cc:237] T e8a548497c4a47feadc568fc4fa549d7 P bc1f5399d01846cba2ce3091796e003f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } }
I20260808 11:37:36.102851 26704 catalog_manager.cc:5495] T e8a548497c4a47feadc568fc4fa549d7 P bc1f5399d01846cba2ce3091796e003f reported cstate change: term changed from 0 to 1, leader changed from <none> to bc1f5399d01846cba2ce3091796e003f (127.24.76.65). New cstate: current_term: 1 leader_uuid: "bc1f5399d01846cba2ce3091796e003f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } health_report { overall_health: UNKNOWN } } }
Aug 08 11:37:36 dist-test-slave-tc2s krb5kdc[26625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189053, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:37:36.234632 27574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:36.216469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36339 (local address 127.0.0.1:59528)
0808 11:37:36.217599 (+ 1130us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:36.217616 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:36.217746 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:36.218170 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:36.218174 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:36.219062 (+ 888us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:37:36.219414 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:36.219424 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:36.221315 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:36.221319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:36.222461 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:36.222474 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:36.223703 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:36.223710 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:36.223942 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:36.224737 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:37:36.224756 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:37:36.227490 (+ 2734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:37:36.231876 (+ 4386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:36.231886 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:36.231892 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:36.232359 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:36.232753 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:36.232758 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:36.232762 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:36.232953 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:36.233508 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:37:36.233517 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:37:36.233703 (+ 186us) client_negotiation.cc:769] Sending connection context
0808 11:37:36.233841 (+ 138us) client_negotiation.cc:240] Negotiation successful
0808 11:37:36.234179 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":963,"thread_start_us":799,"threads_started":1}
I20260808 11:37:36.234743 27576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:36.216536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59528 (local address 127.24.76.126:36339)
0808 11:37:36.217355 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:36.217363 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:36.217698 (+ 335us) server_negotiation.cc:407] Connection header received
0808 11:37:36.217860 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:36.217865 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:36.217983 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:36.218113 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:36.219676 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:36.221146 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:36.222607 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:36.223482 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:36.223611 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:36.227783 (+ 4172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:36.227799 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:36.227804 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:36.227842 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:36.231743 (+ 3901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:36.232459 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:36.232464 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:36.232468 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:36.232656 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:36.233055 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:36.233060 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:36.233064 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:36.233380 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:36.233585 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:36.234001 (+ 416us) server_negotiation.cc:299] Negotiation successful
0808 11:37:36.234309 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":116,"server-negotiator.queue_time_us":650,"thread_start_us":432,"threads_started":1}
I20260808 11:37:36.239826 26704 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:59528:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 11:37:36.241178 26704 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:37:36.248122 27144 tablet_service.cc:1467] Processing CreateTablet for tablet f955f3784cac4ec1bf831993371a26c1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b7b5e3861d9247b086da8ea71f3b3345]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 11:37:36.248113 26921 tablet_service.cc:1467] Processing CreateTablet for tablet f955f3784cac4ec1bf831993371a26c1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b7b5e3861d9247b086da8ea71f3b3345]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 11:37:36.248687 27144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f955f3784cac4ec1bf831993371a26c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:37:36.248731 26921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f955f3784cac4ec1bf831993371a26c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:37:36.251391 27370 tablet_service.cc:1467] Processing CreateTablet for tablet f955f3784cac4ec1bf831993371a26c1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b7b5e3861d9247b086da8ea71f3b3345]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 11:37:36.252017 27370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f955f3784cac4ec1bf831993371a26c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:37:36.255110 27555 tablet_bootstrap.cc:492] T f955f3784cac4ec1bf831993371a26c1 P d0863aa6a60e4e748cf8bab0a94a953b: Bootstrap starting.
I20260808 11:37:36.255887 27557 tablet_bootstrap.cc:492] T f955f3784cac4ec1bf831993371a26c1 P bc1f5399d01846cba2ce3091796e003f: Bootstrap starting.
I20260808 11:37:36.257054 27555 tablet_bootstrap.cc:654] T f955f3784cac4ec1bf831993371a26c1 P d0863aa6a60e4e748cf8bab0a94a953b: Neither blocks nor log segments found. Creating new log.
I20260808 11:37:36.258064 27557 tablet_bootstrap.cc:654] T f955f3784cac4ec1bf831993371a26c1 P bc1f5399d01846cba2ce3091796e003f: Neither blocks nor log segments found. Creating new log.
I20260808 11:37:36.258885 27556 tablet_bootstrap.cc:492] T f955f3784cac4ec1bf831993371a26c1 P 750df1e92cbb4b5198dc9205b5cc125d: Bootstrap starting.
I20260808 11:37:36.260237 27555 tablet_bootstrap.cc:492] T f955f3784cac4ec1bf831993371a26c1 P d0863aa6a60e4e748cf8bab0a94a953b: No bootstrap required, opened a new log
I20260808 11:37:36.260393 27555 ts_tablet_manager.cc:1397] T f955f3784cac4ec1bf831993371a26c1 P d0863aa6a60e4e748cf8bab0a94a953b: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 11:37:36.260805 27556 tablet_bootstrap.cc:654] T f955f3784cac4ec1bf831993371a26c1 P 750df1e92cbb4b5198dc9205b5cc125d: Neither blocks nor log segments found. Creating new log.
I20260808 11:37:36.261003 27557 tablet_bootstrap.cc:492] T f955f3784cac4ec1bf831993371a26c1 P bc1f5399d01846cba2ce3091796e003f: No bootstrap required, opened a new log
I20260808 11:37:36.261168 27557 ts_tablet_manager.cc:1397] T f955f3784cac4ec1bf831993371a26c1 P bc1f5399d01846cba2ce3091796e003f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 11:37:36.261193 27555 raft_consensus.cc:348] T f955f3784cac4ec1bf831993371a26c1 P d0863aa6a60e4e748cf8bab0a94a953b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } }
I20260808 11:37:36.261494 27555 raft_consensus.cc:374] T f955f3784cac4ec1bf831993371a26c1 P d0863aa6a60e4e748cf8bab0a94a953b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:37:36.261582 27555 raft_consensus.cc:729] T f955f3784cac4ec1bf831993371a26c1 P d0863aa6a60e4e748cf8bab0a94a953b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0863aa6a60e4e748cf8bab0a94a953b, State: Initialized, Role: FOLLOWER
I20260808 11:37:36.261933 27557 raft_consensus.cc:348] T f955f3784cac4ec1bf831993371a26c1 P bc1f5399d01846cba2ce3091796e003f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } }
I20260808 11:37:36.261981 27555 consensus_queue.cc:260] T f955f3784cac4ec1bf831993371a26c1 P d0863aa6a60e4e748cf8bab0a94a953b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } }
I20260808 11:37:36.262288 27557 raft_consensus.cc:374] T f955f3784cac4ec1bf831993371a26c1 P bc1f5399d01846cba2ce3091796e003f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:37:36.262374 27557 raft_consensus.cc:729] T f955f3784cac4ec1bf831993371a26c1 P bc1f5399d01846cba2ce3091796e003f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc1f5399d01846cba2ce3091796e003f, State: Initialized, Role: FOLLOWER
I20260808 11:37:36.262773 27557 consensus_queue.cc:260] T f955f3784cac4ec1bf831993371a26c1 P bc1f5399d01846cba2ce3091796e003f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } }
I20260808 11:37:36.262970 27519 tablet_replica.cc:439] T f955f3784cac4ec1bf831993371a26c1 P d0863aa6a60e4e748cf8bab0a94a953b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } } }
I20260808 11:37:36.263378 27557 ts_tablet_manager.cc:1428] T f955f3784cac4ec1bf831993371a26c1 P bc1f5399d01846cba2ce3091796e003f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 11:37:36.263294 27523 tablet_replica.cc:439] T f955f3784cac4ec1bf831993371a26c1 P bc1f5399d01846cba2ce3091796e003f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } } }
I20260808 11:37:36.263455 27519 tablet_replica.cc:442] T f955f3784cac4ec1bf831993371a26c1 P d0863aa6a60e4e748cf8bab0a94a953b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:37:36.263618 27523 tablet_replica.cc:442] T f955f3784cac4ec1bf831993371a26c1 P bc1f5399d01846cba2ce3091796e003f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:37:36.262648 27555 ts_tablet_manager.cc:1428] T f955f3784cac4ec1bf831993371a26c1 P d0863aa6a60e4e748cf8bab0a94a953b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 11:37:36.264822 27556 tablet_bootstrap.cc:492] T f955f3784cac4ec1bf831993371a26c1 P 750df1e92cbb4b5198dc9205b5cc125d: No bootstrap required, opened a new log
I20260808 11:37:36.265069 27556 ts_tablet_manager.cc:1397] T f955f3784cac4ec1bf831993371a26c1 P 750df1e92cbb4b5198dc9205b5cc125d: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.001s
I20260808 11:37:36.266013 27556 raft_consensus.cc:348] T f955f3784cac4ec1bf831993371a26c1 P 750df1e92cbb4b5198dc9205b5cc125d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } }
I20260808 11:37:36.266284 27556 raft_consensus.cc:374] T f955f3784cac4ec1bf831993371a26c1 P 750df1e92cbb4b5198dc9205b5cc125d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:37:36.266417 27556 raft_consensus.cc:729] T f955f3784cac4ec1bf831993371a26c1 P 750df1e92cbb4b5198dc9205b5cc125d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 750df1e92cbb4b5198dc9205b5cc125d, State: Initialized, Role: FOLLOWER
I20260808 11:37:36.266642 27556 consensus_queue.cc:260] T f955f3784cac4ec1bf831993371a26c1 P 750df1e92cbb4b5198dc9205b5cc125d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } }
I20260808 11:37:36.267025 27520 tablet_replica.cc:439] T f955f3784cac4ec1bf831993371a26c1 P 750df1e92cbb4b5198dc9205b5cc125d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } } }
I20260808 11:37:36.267339 27520 tablet_replica.cc:442] T f955f3784cac4ec1bf831993371a26c1 P 750df1e92cbb4b5198dc9205b5cc125d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:37:36.267179 27556 ts_tablet_manager.cc:1428] T f955f3784cac4ec1bf831993371a26c1 P 750df1e92cbb4b5198dc9205b5cc125d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 11:37:36.296838 27519 raft_consensus.cc:482] T f955f3784cac4ec1bf831993371a26c1 P d0863aa6a60e4e748cf8bab0a94a953b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:37:36.297019 27519 raft_consensus.cc:504] T f955f3784cac4ec1bf831993371a26c1 P d0863aa6a60e4e748cf8bab0a94a953b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } }
I20260808 11:37:36.298467 27519 leader_election.cc:288] T f955f3784cac4ec1bf831993371a26c1 P d0863aa6a60e4e748cf8bab0a94a953b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc1f5399d01846cba2ce3091796e003f (127.24.76.65:43749), 750df1e92cbb4b5198dc9205b5cc125d (127.24.76.67:37105)
I20260808 11:37:36.317574 27586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:36.298901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:38109 (local address 127.24.76.65:43749)
0808 11:37:36.311303 (+ 12402us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:36.311308 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:36.311334 (+ 26us) server_negotiation.cc:407] Connection header received
0808 11:37:36.311394 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:36.311398 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:36.311454 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:36.311535 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:37:36.312436 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:36.313652 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:36.315516 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:36.316512 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:36.316664 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:36.316741 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:36.317061 (+ 320us) server_negotiation.cc:299] Negotiation successful
0808 11:37:36.317194 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12303,"thread_start_us":12193,"threads_started":1}
I20260808 11:37:36.317623 27581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:36.298638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:43749 (local address 127.24.76.66:38109)
0808 11:37:36.310667 (+ 12029us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:36.310691 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:36.310835 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:36.311600 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:36.311605 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:36.311638 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:37:36.312136 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:36.312150 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:36.313806 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:36.313810 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:36.315341 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:36.315354 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:36.316702 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:36.316709 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:36.316909 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:36.316935 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 11:37:36.317017 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 11:37:36.317156 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11902,"mutex_wait_us":23,"thread_start_us":12643,"threads_started":1}
I20260808 11:37:36.318041 27585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:36.298824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:53531 (local address 127.24.76.67:37105)
0808 11:37:36.310992 (+ 12168us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:36.310999 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:36.311655 (+ 656us) server_negotiation.cc:407] Connection header received
0808 11:37:36.311947 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:36.311953 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:36.312021 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:36.312097 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:37:36.312913 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:36.313886 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:36.315934 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:36.316960 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:36.317108 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:36.317177 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:36.317490 (+ 313us) server_negotiation.cc:299] Negotiation successful
0808 11:37:36.317623 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12005,"thread_start_us":11906,"threads_started":1}
I20260808 11:37:36.318311 27583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:36.298696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:37105 (local address 127.24.76.66:53531)
0808 11:37:36.311539 (+ 12843us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:36.311578 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:36.311772 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:36.312152 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:36.312156 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:36.312183 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:37:36.312738 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:36.312755 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:36.314058 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:36.314063 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:36.315775 (+ 1712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:36.315789 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:36.317152 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:36.317159 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:36.317341 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:36.317365 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 11:37:36.317442 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 11:37:36.317574 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12714,"mutex_wait_us":24,"thread_start_us":11755,"threads_started":1}
I20260808 11:37:36.318475 26955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f955f3784cac4ec1bf831993371a26c1" candidate_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc1f5399d01846cba2ce3091796e003f" is_pre_election: true
I20260808 11:37:36.318760 26955 raft_consensus.cc:2454] T f955f3784cac4ec1bf831993371a26c1 P bc1f5399d01846cba2ce3091796e003f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0863aa6a60e4e748cf8bab0a94a953b in term 0.
I20260808 11:37:36.318931 27402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f955f3784cac4ec1bf831993371a26c1" candidate_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "750df1e92cbb4b5198dc9205b5cc125d" is_pre_election: true
I20260808 11:37:36.319211 27402 raft_consensus.cc:2454] T f955f3784cac4ec1bf831993371a26c1 P 750df1e92cbb4b5198dc9205b5cc125d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0863aa6a60e4e748cf8bab0a94a953b in term 0.
I20260808 11:37:36.319314 27062 leader_election.cc:302] T f955f3784cac4ec1bf831993371a26c1 P d0863aa6a60e4e748cf8bab0a94a953b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc1f5399d01846cba2ce3091796e003f, d0863aa6a60e4e748cf8bab0a94a953b; no voters:
I20260808 11:37:36.319746 27519 raft_consensus.cc:2781] T f955f3784cac4ec1bf831993371a26c1 P d0863aa6a60e4e748cf8bab0a94a953b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:37:36.319918 27519 raft_consensus.cc:482] T f955f3784cac4ec1bf831993371a26c1 P d0863aa6a60e4e748cf8bab0a94a953b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:37:36.320020 27519 raft_consensus.cc:3037] T f955f3784cac4ec1bf831993371a26c1 P d0863aa6a60e4e748cf8bab0a94a953b [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:37:36.322772 27519 raft_consensus.cc:504] T f955f3784cac4ec1bf831993371a26c1 P d0863aa6a60e4e748cf8bab0a94a953b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } }
I20260808 11:37:36.323354 27519 leader_election.cc:288] T f955f3784cac4ec1bf831993371a26c1 P d0863aa6a60e4e748cf8bab0a94a953b [CANDIDATE]: Term 1 election: Requested vote from peers bc1f5399d01846cba2ce3091796e003f (127.24.76.65:43749), 750df1e92cbb4b5198dc9205b5cc125d (127.24.76.67:37105)
I20260808 11:37:36.323813 26955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f955f3784cac4ec1bf831993371a26c1" candidate_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc1f5399d01846cba2ce3091796e003f"
I20260808 11:37:36.323877 27402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f955f3784cac4ec1bf831993371a26c1" candidate_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "750df1e92cbb4b5198dc9205b5cc125d"
I20260808 11:37:36.324020 26955 raft_consensus.cc:3037] T f955f3784cac4ec1bf831993371a26c1 P bc1f5399d01846cba2ce3091796e003f [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:37:36.324201 27402 raft_consensus.cc:3037] T f955f3784cac4ec1bf831993371a26c1 P 750df1e92cbb4b5198dc9205b5cc125d [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:37:36.326495 26955 raft_consensus.cc:2454] T f955f3784cac4ec1bf831993371a26c1 P bc1f5399d01846cba2ce3091796e003f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0863aa6a60e4e748cf8bab0a94a953b in term 1.
I20260808 11:37:36.326612 27402 raft_consensus.cc:2454] T f955f3784cac4ec1bf831993371a26c1 P 750df1e92cbb4b5198dc9205b5cc125d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0863aa6a60e4e748cf8bab0a94a953b in term 1.
I20260808 11:37:36.326920 27062 leader_election.cc:302] T f955f3784cac4ec1bf831993371a26c1 P d0863aa6a60e4e748cf8bab0a94a953b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc1f5399d01846cba2ce3091796e003f, d0863aa6a60e4e748cf8bab0a94a953b; no voters:
I20260808 11:37:36.327236 27519 raft_consensus.cc:2781] T f955f3784cac4ec1bf831993371a26c1 P d0863aa6a60e4e748cf8bab0a94a953b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:37:36.327791 27519 raft_consensus.cc:686] T f955f3784cac4ec1bf831993371a26c1 P d0863aa6a60e4e748cf8bab0a94a953b [term 1 LEADER]: Becoming Leader. State: Replica: d0863aa6a60e4e748cf8bab0a94a953b, State: Running, Role: LEADER
I20260808 11:37:36.328154 27519 consensus_queue.cc:237] T f955f3784cac4ec1bf831993371a26c1 P d0863aa6a60e4e748cf8bab0a94a953b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } }
I20260808 11:37:36.330374 27587 tablet_replica.cc:439] T f955f3784cac4ec1bf831993371a26c1 P d0863aa6a60e4e748cf8bab0a94a953b: TxnStatusTablet state changed. Reason: New leader d0863aa6a60e4e748cf8bab0a94a953b. Latest consensus state: current_term: 1 leader_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } } }
I20260808 11:37:36.330834 27587 tablet_replica.cc:442] T f955f3784cac4ec1bf831993371a26c1 P d0863aa6a60e4e748cf8bab0a94a953b: This TxnStatusTablet replica's current role is: LEADER
I20260808 11:37:36.331897 27589 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 11:37:36.331940 26704 catalog_manager.cc:5495] T f955f3784cac4ec1bf831993371a26c1 P d0863aa6a60e4e748cf8bab0a94a953b reported cstate change: term changed from 0 to 1, leader changed from <none> to d0863aa6a60e4e748cf8bab0a94a953b (127.24.76.66). New cstate: current_term: 1 leader_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } health_report { overall_health: UNKNOWN } } }
I20260808 11:37:36.332309 27589 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 11:37:36.365044 27574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:36.357179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:44735 (local address 127.0.0.1:48918)
0808 11:37:36.357464 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:36.357499 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:36.357658 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:36.358552 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:36.358555 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:36.358565 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 11:37:36.358809 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:36.358818 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:36.360535 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:36.360539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:36.361833 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:36.361846 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:36.363010 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:36.363014 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:36.363246 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:36.363336 (+ 90us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 11:37:36.364563 (+ 1227us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 11:37:36.364581 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 11:37:36.364635 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 11:37:36.364779 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 11:37:36.365221 27599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:36.357258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48918 (local address 127.24.76.66:44735)
0808 11:37:36.358108 (+ 850us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:36.358114 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:36.358140 (+ 26us) server_negotiation.cc:407] Connection header received
0808 11:37:36.358229 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:36.358234 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:36.358347 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:36.358539 (+ 192us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:37:36.359026 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:36.360427 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:36.361981 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:36.362785 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:36.362939 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:36.363558 (+ 619us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:37:36.364462 (+ 904us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:37:36.364608 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:36.364676 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 11:37:36.364760 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":428,"threads_started":1}
I20260808 11:37:36.370219 27402 raft_consensus.cc:1261] T f955f3784cac4ec1bf831993371a26c1 P 750df1e92cbb4b5198dc9205b5cc125d [term 1 FOLLOWER]: Refusing update from remote peer d0863aa6a60e4e748cf8bab0a94a953b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:37:36.371129 26955 raft_consensus.cc:1261] T f955f3784cac4ec1bf831993371a26c1 P bc1f5399d01846cba2ce3091796e003f [term 1 FOLLOWER]: Refusing update from remote peer d0863aa6a60e4e748cf8bab0a94a953b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:37:36.371006 27587 consensus_queue.cc:1035] T f955f3784cac4ec1bf831993371a26c1 P d0863aa6a60e4e748cf8bab0a94a953b [LEADER]: Connected to new peer: Peer: permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:37:36.372004 27587 consensus_queue.cc:1035] T f955f3784cac4ec1bf831993371a26c1 P d0863aa6a60e4e748cf8bab0a94a953b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:37:36.378944 27520 tablet_replica.cc:439] T f955f3784cac4ec1bf831993371a26c1 P 750df1e92cbb4b5198dc9205b5cc125d: TxnStatusTablet state changed. Reason: New leader d0863aa6a60e4e748cf8bab0a94a953b. Latest consensus state: current_term: 1 leader_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } } }
I20260808 11:37:36.379472 27520 tablet_replica.cc:442] T f955f3784cac4ec1bf831993371a26c1 P 750df1e92cbb4b5198dc9205b5cc125d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:37:36.382535 27601 mvcc.cc:205] Tried to move back new op lower bound from 7316230374884646912 to 7316230374724341760. Current Snapshot: MvccSnapshot[applied={T|T < 7316230374884646912}]
I20260808 11:37:36.383407 27520 tablet_replica.cc:439] T f955f3784cac4ec1bf831993371a26c1 P 750df1e92cbb4b5198dc9205b5cc125d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } } }
I20260808 11:37:36.383642 27520 tablet_replica.cc:442] T f955f3784cac4ec1bf831993371a26c1 P 750df1e92cbb4b5198dc9205b5cc125d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:37:36.387883 27523 tablet_replica.cc:439] T f955f3784cac4ec1bf831993371a26c1 P bc1f5399d01846cba2ce3091796e003f: TxnStatusTablet state changed. Reason: New leader d0863aa6a60e4e748cf8bab0a94a953b. Latest consensus state: current_term: 1 leader_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } } }
I20260808 11:37:36.388248 27523 tablet_replica.cc:442] T f955f3784cac4ec1bf831993371a26c1 P bc1f5399d01846cba2ce3091796e003f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:37:36.389114 27587 tablet_replica.cc:439] T f955f3784cac4ec1bf831993371a26c1 P d0863aa6a60e4e748cf8bab0a94a953b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } } }
I20260808 11:37:36.389374 27587 tablet_replica.cc:442] T f955f3784cac4ec1bf831993371a26c1 P d0863aa6a60e4e748cf8bab0a94a953b: This TxnStatusTablet replica's current role is: LEADER
I20260808 11:37:36.392565 27587 tablet_replica.cc:439] T f955f3784cac4ec1bf831993371a26c1 P d0863aa6a60e4e748cf8bab0a94a953b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } } }
I20260808 11:37:36.392830 27587 tablet_replica.cc:442] T f955f3784cac4ec1bf831993371a26c1 P d0863aa6a60e4e748cf8bab0a94a953b: This TxnStatusTablet replica's current role is: LEADER
I20260808 11:37:36.395859 27523 tablet_replica.cc:439] T f955f3784cac4ec1bf831993371a26c1 P bc1f5399d01846cba2ce3091796e003f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc1f5399d01846cba2ce3091796e003f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43749 } } peers { permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 } } peers { permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 } } }
I20260808 11:37:36.396143 27523 tablet_replica.cc:442] T f955f3784cac4ec1bf831993371a26c1 P bc1f5399d01846cba2ce3091796e003f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:37:36.413914 27622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:36.407325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:44735 (local address 127.24.76.65:60981)
0808 11:37:36.408019 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:36.408034 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:36.408148 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:36.408491 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:36.408494 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:36.408504 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 11:37:36.408893 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:36.408903 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:36.410531 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:36.410535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:36.411658 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:36.411671 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:36.412787 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:36.412795 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:36.412932 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:36.412985 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 11:37:36.413402 (+ 417us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 11:37:36.413411 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 11:37:36.413468 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 11:37:36.413567 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":367,"threads_started":1}
I20260808 11:37:36.413959 27599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:36.407452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:60981 (local address 127.24.76.66:44735)
0808 11:37:36.407623 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:36.407629 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:36.408116 (+ 487us) server_negotiation.cc:407] Connection header received
0808 11:37:36.408266 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:36.408272 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:36.408382 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:36.408487 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:37:36.409031 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:36.410392 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:36.411796 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:36.412618 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:36.412761 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:36.413155 (+ 394us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:37:36.413294 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:37:36.413388 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:36.413505 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 11:37:36.413584 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 11:37:36.425213 27402 raft_consensus.cc:1261] T e8a548497c4a47feadc568fc4fa549d7 P 750df1e92cbb4b5198dc9205b5cc125d [term 1 FOLLOWER]: Refusing update from remote peer bc1f5399d01846cba2ce3091796e003f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:37:36.425303 27181 raft_consensus.cc:1261] T e8a548497c4a47feadc568fc4fa549d7 P d0863aa6a60e4e748cf8bab0a94a953b [term 1 FOLLOWER]: Refusing update from remote peer bc1f5399d01846cba2ce3091796e003f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:37:36.425856 27523 consensus_queue.cc:1035] T e8a548497c4a47feadc568fc4fa549d7 P bc1f5399d01846cba2ce3091796e003f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0863aa6a60e4e748cf8bab0a94a953b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:37:36.426177 27531 consensus_queue.cc:1035] T e8a548497c4a47feadc568fc4fa549d7 P bc1f5399d01846cba2ce3091796e003f [LEADER]: Connected to new peer: Peer: permanent_uuid: "750df1e92cbb4b5198dc9205b5cc125d" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 37105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:37:36.464197 27586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:36.455981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:48377 (local address 127.24.76.65:43749)
0808 11:37:36.456192 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:36.456199 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:36.457215 (+ 1016us) server_negotiation.cc:407] Connection header received
0808 11:37:36.457452 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:36.457458 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:36.457542 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:36.457630 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:37:36.458719 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:36.459520 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:36.461931 (+ 2411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:36.462598 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:36.462709 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:36.463405 (+ 696us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:37:36.463521 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:37:36.463612 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:36.463813 (+ 201us) server_negotiation.cc:299] Negotiation successful
0808 11:37:36.463915 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 11:37:36.464599 27633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:36.456122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:43749 (local address 127.24.76.66:48377)
0808 11:37:36.457107 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:36.457127 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:36.457320 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:36.457665 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:36.457671 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:36.457685 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 11:37:36.458555 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:36.458571 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:36.459642 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:36.459648 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:36.461711 (+ 2063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:36.461728 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:36.462816 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:36.462824 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:36.463100 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:36.463192 (+ 92us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 11:37:36.463690 (+ 498us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 11:37:36.463703 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 11:37:36.463790 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 11:37:36.463927 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":760,"thread_start_us":575,"threads_started":1}
I20260808 11:37:36.508314 26704 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59526:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:37:36.508555 26704 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59526:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:37:36.511488 26704 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 974ca9bbdcd44562b076f7c2628d921c: Sending DeleteTablet for 3 replicas of tablet e8a548497c4a47feadc568fc4fa549d7
I20260808 11:37:36.512282 26921 tablet_service.cc:1514] Processing DeleteTablet for tablet e8a548497c4a47feadc568fc4fa549d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:37:36 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:53886
I20260808 11:37:36.512657 27144 tablet_service.cc:1514] Processing DeleteTablet for tablet e8a548497c4a47feadc568fc4fa549d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:37:36 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:48902
I20260808 11:37:36.513952 27370 tablet_service.cc:1514] Processing DeleteTablet for tablet e8a548497c4a47feadc568fc4fa549d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:37:36 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:60018
I20260808 11:37:36.521378 27639 tablet_replica.cc:330] T e8a548497c4a47feadc568fc4fa549d7 P 750df1e92cbb4b5198dc9205b5cc125d: stopping tablet replica
I20260808 11:37:36.521579 27639 raft_consensus.cc:2229] T e8a548497c4a47feadc568fc4fa549d7 P 750df1e92cbb4b5198dc9205b5cc125d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:37:36.523000 27639 raft_consensus.cc:2258] T e8a548497c4a47feadc568fc4fa549d7 P 750df1e92cbb4b5198dc9205b5cc125d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:37:36.524411 27639 ts_tablet_manager.cc:1905] T e8a548497c4a47feadc568fc4fa549d7 P 750df1e92cbb4b5198dc9205b5cc125d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:37:36.532907 27639 ts_tablet_manager.cc:1918] T e8a548497c4a47feadc568fc4fa549d7 P 750df1e92cbb4b5198dc9205b5cc125d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 11:37:36.533042 27639 log.cc:1196] T e8a548497c4a47feadc568fc4fa549d7 P 750df1e92cbb4b5198dc9205b5cc125d: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-2/wal/wals/e8a548497c4a47feadc568fc4fa549d7
I20260808 11:37:36.533679 27639 ts_tablet_manager.cc:1939] T e8a548497c4a47feadc568fc4fa549d7 P 750df1e92cbb4b5198dc9205b5cc125d: Deleting consensus metadata
I20260808 11:37:36.534760 26679 catalog_manager.cc:4841] TS 750df1e92cbb4b5198dc9205b5cc125d (127.24.76.67:37105): tablet e8a548497c4a47feadc568fc4fa549d7 (table test-table [id=3fe3e7334d024caaa0fe4a5ded36112e]) successfully deleted
I20260808 11:37:36.539319 27638 tablet_replica.cc:330] T e8a548497c4a47feadc568fc4fa549d7 P d0863aa6a60e4e748cf8bab0a94a953b: stopping tablet replica
I20260808 11:37:36.539558 27638 raft_consensus.cc:2229] T e8a548497c4a47feadc568fc4fa549d7 P d0863aa6a60e4e748cf8bab0a94a953b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:37:36.539994 27638 raft_consensus.cc:2258] T e8a548497c4a47feadc568fc4fa549d7 P d0863aa6a60e4e748cf8bab0a94a953b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:37:36.540719 27636 tablet_replica.cc:330] T e8a548497c4a47feadc568fc4fa549d7 P bc1f5399d01846cba2ce3091796e003f: stopping tablet replica
I20260808 11:37:36.540926 27636 raft_consensus.cc:2229] T e8a548497c4a47feadc568fc4fa549d7 P bc1f5399d01846cba2ce3091796e003f [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:37:36.541245 27638 ts_tablet_manager.cc:1905] T e8a548497c4a47feadc568fc4fa549d7 P d0863aa6a60e4e748cf8bab0a94a953b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:37:36.541386 27636 raft_consensus.cc:2258] T e8a548497c4a47feadc568fc4fa549d7 P bc1f5399d01846cba2ce3091796e003f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:37:36.542536 27636 ts_tablet_manager.cc:1905] T e8a548497c4a47feadc568fc4fa549d7 P bc1f5399d01846cba2ce3091796e003f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:37:36.549762 27638 ts_tablet_manager.cc:1918] T e8a548497c4a47feadc568fc4fa549d7 P d0863aa6a60e4e748cf8bab0a94a953b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 11:37:36.549983 27638 log.cc:1196] T e8a548497c4a47feadc568fc4fa549d7 P d0863aa6a60e4e748cf8bab0a94a953b: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-1/wal/wals/e8a548497c4a47feadc568fc4fa549d7
I20260808 11:37:36.550195 27636 ts_tablet_manager.cc:1918] T e8a548497c4a47feadc568fc4fa549d7 P bc1f5399d01846cba2ce3091796e003f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 11:37:36.550328 27636 log.cc:1196] T e8a548497c4a47feadc568fc4fa549d7 P bc1f5399d01846cba2ce3091796e003f: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786189042529715-24881-0/minicluster-data/ts-0/wal/wals/e8a548497c4a47feadc568fc4fa549d7
I20260808 11:37:36.550661 27638 ts_tablet_manager.cc:1939] T e8a548497c4a47feadc568fc4fa549d7 P d0863aa6a60e4e748cf8bab0a94a953b: Deleting consensus metadata
I20260808 11:37:36.550887 27636 ts_tablet_manager.cc:1939] T e8a548497c4a47feadc568fc4fa549d7 P bc1f5399d01846cba2ce3091796e003f: Deleting consensus metadata
I20260808 11:37:36.551986 26676 catalog_manager.cc:4841] TS d0863aa6a60e4e748cf8bab0a94a953b (127.24.76.66:44735): tablet e8a548497c4a47feadc568fc4fa549d7 (table test-table [id=3fe3e7334d024caaa0fe4a5ded36112e]) successfully deleted
I20260808 11:37:36.552070 26677 catalog_manager.cc:4841] TS bc1f5399d01846cba2ce3091796e003f (127.24.76.65:43749): tablet e8a548497c4a47feadc568fc4fa549d7 (table test-table [id=3fe3e7334d024caaa0fe4a5ded36112e]) successfully deleted
Aug 08 11:37:36 dist-test-slave-tc2s krb5kdc[26625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189055, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.76.65@KRBTEST.COM
I20260808 11:37:36.570785 27586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:36.539759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53900 (local address 127.24.76.65:43749)
0808 11:37:36.540008 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:36.540014 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:36.552496 (+ 12482us) server_negotiation.cc:407] Connection header received
0808 11:37:36.552633 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:36.552639 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:36.552722 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:36.552825 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:36.554606 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:36.555981 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:36.557611 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:36.558393 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:36.558518 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:36.563006 (+ 4488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:36.563019 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:36.563036 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:36.563090 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:36.567366 (+ 4276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:36.568139 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:36.568149 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:36.568153 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:36.568438 (+ 285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:36.568850 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:36.568855 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:36.568858 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:36.569342 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:36.569666 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:36.569935 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 11:37:36.570152 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
W20260808 11:37:36.571652 27023 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53900
I20260808 11:37:36.589078 27576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:36.574591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59540 (local address 127.24.76.126:36339)
0808 11:37:36.574817 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:36.574822 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:36.574907 (+ 85us) server_negotiation.cc:407] Connection header received
0808 11:37:36.575045 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:36.575051 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:36.575133 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:36.575237 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:36.576637 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:36.577772 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:36.579183 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:36.579946 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:36.580074 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:36.582891 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:36.582896 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:36.582900 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:36.582931 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:36.586054 (+ 3123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:36.586747 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:36.586753 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:36.586755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:36.586934 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:36.587268 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:36.587271 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:36.587273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:36.587710 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:36.587975 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:36.588449 (+ 474us) server_negotiation.cc:299] Negotiation successful
0808 11:37:36.588671 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
W20260808 11:37:36.589622 26704 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59540
I20260808 11:37:36.592334 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 26814
I20260808 11:37:36.618566 27586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:36.593429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53902 (local address 127.24.76.65:43749)
0808 11:37:36.593636 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:36.593643 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:36.606349 (+ 12706us) server_negotiation.cc:407] Connection header received
0808 11:37:36.606491 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:36.606496 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:36.606576 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:36.606658 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:36.607924 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:36.608620 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:36.609868 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:36.610336 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:36.610454 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:36.613227 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:36.613232 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:36.613235 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:36.613267 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:36.615990 (+ 2723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:36.616650 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:36.616655 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:36.616658 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:36.616778 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:36.617119 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:36.617124 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:36.617128 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:36.617444 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:36.617699 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:36.618030 (+ 331us) server_negotiation.cc:299] Negotiation successful
0808 11:37:36.618243 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260808 11:37:36.619251 27023 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53902
W20260808 11:37:36.619805 24881 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 26814: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:37:36.619987 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 26814
I20260808 11:37:36.663591 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 27038
Aug 08 11:37:36 dist-test-slave-tc2s krb5kdc[26625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189055, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.76.66@KRBTEST.COM
I20260808 11:37:36.681376 27599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:36.664446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48932 (local address 127.24.76.66:44735)
0808 11:37:36.664684 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:36.664690 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:36.664735 (+ 45us) server_negotiation.cc:407] Connection header received
0808 11:37:36.664803 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:36.664809 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:36.664893 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:36.665021 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:36.666209 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:36.667507 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:36.668710 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:36.669494 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:36.669625 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:36.674654 (+ 5029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:36.674669 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:36.674684 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:36.674729 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:36.678598 (+ 3869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:36.679214 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:36.679223 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:36.679227 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:36.679395 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:36.679721 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:36.679727 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:36.679730 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:36.680199 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:36.680552 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:36.680715 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 11:37:36.680953 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260808 11:37:36.682180 27244 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48932
W20260808 11:37:36.682835 24881 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 27038: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:37:36.683027 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 27038
I20260808 11:37:36.710395 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 27257
Aug 08 11:37:36 dist-test-slave-tc2s krb5kdc[26625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189055, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.76.67@KRBTEST.COM
I20260808 11:37:36.724552 27585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:36.711120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60020 (local address 127.24.76.67:37105)
0808 11:37:36.711315 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:36.711321 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:36.711342 (+ 21us) server_negotiation.cc:407] Connection header received
0808 11:37:36.711514 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:36.711518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:36.711583 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:36.711679 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:36.712921 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:36.713630 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:36.714898 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:36.715312 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:36.715420 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:36.719389 (+ 3969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:36.719405 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:36.719422 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:36.719463 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:36.722004 (+ 2541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:36.722624 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:36.722631 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:36.722633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:36.722767 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:36.723112 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:36.723116 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:36.723118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:36.723452 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:36.723754 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:36.724041 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 11:37:36.724275 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260808 11:37:36.725217 27477 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60020
W20260808 11:37:36.725804 24881 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 27257: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:37:36.725975 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 27257
I20260808 11:37:36.750470 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 26647
I20260808 11:37:36.751556 26764 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:37:36.867086 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 26647
2026-08-08T11:37:36Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5164 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:37:36 dist-test-slave-tc2s krb5kdc[27653](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:37:36 dist-test-slave-tc2s krb5kdc[27653](info): set up 2 sockets
Aug 08 11:37:36 dist-test-slave-tc2s krb5kdc[27653](info): commencing operation
krb5kdc: starting...
W20260808 11:37:38.062873 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.001s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:37:38 dist-test-slave-tc2s krb5kdc[27653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:37:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:37:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
I20260808 11:37:38.248631 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:45521
--webserver_interface=127.24.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:35601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:45521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:37:38.565243 27671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:37:38.565537 27671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:37:38.565671 27671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:37:38.572692 27671 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:37:38.572804 27671 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 11:37:38.572871 27671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:37:38.572957 27671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:37:38.573022 27671 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:37:38.573104 27671 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:37:38.582674 27671 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:35601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:45521
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:45521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:37:38.583854 27671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:37:38.585116 27671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:37:38.596081 27680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:38.596174 27681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:38.596140 27688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:37:38.597043 27671 server_base.cc:1034] running on GCE node
I20260808 11:37:38.597764 27671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:37:38.599100 27671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:37:38.600324 27671 hybrid_clock.cc:648] HybridClock initialized: now 1786189058600243 us; error 44 us; skew 500 ppm
I20260808 11:37:38.600629 27671 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:37:38 dist-test-slave-tc2s krb5kdc[27653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189058, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:37:38.603542 27671 init.cc:377] Logged in from keytab as kudu/127.24.76.126@KRBTEST.COM (short username kudu)
I20260808 11:37:38.604938 27671 webserver.cc:460] Webserver started at http://127.24.76.126:37763/ using document root <none> and password file <none>
I20260808 11:37:38.605608 27671 fs_manager.cc:359] Metadata directory not provided
I20260808 11:37:38.605705 27671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:37:38.606096 27671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:37:38.608964 27671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "1d2e375a1ba543f0ae765fab7a56ca28"
format_stamp: "Formatted at 2026-08-08 11:37:38 on dist-test-slave-tc2s"
server_key: "a40ddf4ff080c8e66582d909547f4c43"
server_key_iv: "8a99c6e95a5a8a7380f3e1033976fde5"
server_key_version: "encryptionkey@0"
I20260808 11:37:38.609724 27671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "1d2e375a1ba543f0ae765fab7a56ca28"
format_stamp: "Formatted at 2026-08-08 11:37:38 on dist-test-slave-tc2s"
server_key: "a40ddf4ff080c8e66582d909547f4c43"
server_key_iv: "8a99c6e95a5a8a7380f3e1033976fde5"
server_key_version: "encryptionkey@0"
I20260808 11:37:38.614650 27671 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260808 11:37:38.617771 27697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:38.619171 27671 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 11:37:38.619342 27671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "1d2e375a1ba543f0ae765fab7a56ca28"
format_stamp: "Formatted at 2026-08-08 11:37:38 on dist-test-slave-tc2s"
server_key: "a40ddf4ff080c8e66582d909547f4c43"
server_key_iv: "8a99c6e95a5a8a7380f3e1033976fde5"
server_key_version: "encryptionkey@0"
I20260808 11:37:38.619542 27671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:37:38.632901 27671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:37:38.636454 27671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:37:38.636703 27671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:37:38.655839 27671 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:45521
I20260808 11:37:38.655867 27788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:45521 every 8 connection(s)
I20260808 11:37:38.657109 27671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:37:38.660742 27791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:37:38.664115 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 27671
I20260808 11:37:38.664296 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:37:38.664913 24881 external_mini_cluster.cc:1428] Setting key 8e27f565daaae2cc4fa8f3237e556669
I20260808 11:37:38.672914 27791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d2e375a1ba543f0ae765fab7a56ca28: Bootstrap starting.
I20260808 11:37:38.676584 27791 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d2e375a1ba543f0ae765fab7a56ca28: Neither blocks nor log segments found. Creating new log.
I20260808 11:37:38.677834 27791 log.cc:824] T 00000000000000000000000000000000 P 1d2e375a1ba543f0ae765fab7a56ca28: Log is configured to *not* fsync() on all Append() calls
Aug 08 11:37:38 dist-test-slave-tc2s krb5kdc[27653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:37:38.680591 27791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d2e375a1ba543f0ae765fab7a56ca28: No bootstrap required, opened a new log
I20260808 11:37:38.687029 27796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:38.666749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60414 (local address 127.24.76.126:45521)
0808 11:37:38.667477 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:38.667488 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:38.667529 (+ 41us) server_negotiation.cc:407] Connection header received
0808 11:37:38.669102 (+ 1573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:38.669128 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:38.669505 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:38.669823 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:38.671570 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:38.672822 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:38.674548 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:38.675227 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:38.675352 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:38.680086 (+ 4734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:38.680094 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:38.680107 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:38.680142 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:38.683135 (+ 2993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:38.683797 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:38.683806 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:38.683812 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:38.683983 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:38.684353 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:38.684356 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:38.684358 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:38.684886 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:38.685179 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:38.685627 (+ 448us) server_negotiation.cc:299] Negotiation successful
0808 11:37:38.686059 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":327,"threads_started":1}
I20260808 11:37:38.688063 27791 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d2e375a1ba543f0ae765fab7a56ca28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d2e375a1ba543f0ae765fab7a56ca28" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 45521 } }
I20260808 11:37:38.688395 27791 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d2e375a1ba543f0ae765fab7a56ca28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:37:38.688503 27791 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d2e375a1ba543f0ae765fab7a56ca28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d2e375a1ba543f0ae765fab7a56ca28, State: Initialized, Role: FOLLOWER
I20260808 11:37:38.689054 27791 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d2e375a1ba543f0ae765fab7a56ca28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d2e375a1ba543f0ae765fab7a56ca28" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 45521 } }
I20260808 11:37:38.689287 27791 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d2e375a1ba543f0ae765fab7a56ca28 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:37:38.689426 27791 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d2e375a1ba543f0ae765fab7a56ca28 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:37:38.689610 27791 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d2e375a1ba543f0ae765fab7a56ca28 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:37:38.691869 27791 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d2e375a1ba543f0ae765fab7a56ca28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d2e375a1ba543f0ae765fab7a56ca28" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 45521 } }
I20260808 11:37:38.692307 27791 leader_election.cc:302] T 00000000000000000000000000000000 P 1d2e375a1ba543f0ae765fab7a56ca28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d2e375a1ba543f0ae765fab7a56ca28; no voters:
I20260808 11:37:38.692792 27791 leader_election.cc:288] T 00000000000000000000000000000000 P 1d2e375a1ba543f0ae765fab7a56ca28 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:37:38.692910 27798 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d2e375a1ba543f0ae765fab7a56ca28 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:37:38.693553 27798 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d2e375a1ba543f0ae765fab7a56ca28 [term 1 LEADER]: Becoming Leader. State: Replica: 1d2e375a1ba543f0ae765fab7a56ca28, State: Running, Role: LEADER
I20260808 11:37:38.694072 27798 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d2e375a1ba543f0ae765fab7a56ca28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d2e375a1ba543f0ae765fab7a56ca28" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 45521 } }
I20260808 11:37:38.694615 27791 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d2e375a1ba543f0ae765fab7a56ca28 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:37:38.696596 27800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d2e375a1ba543f0ae765fab7a56ca28 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d2e375a1ba543f0ae765fab7a56ca28. Latest consensus state: current_term: 1 leader_uuid: "1d2e375a1ba543f0ae765fab7a56ca28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d2e375a1ba543f0ae765fab7a56ca28" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 45521 } } }
I20260808 11:37:38.697007 27800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d2e375a1ba543f0ae765fab7a56ca28 [sys.catalog]: This master's current role is: LEADER
I20260808 11:37:38.697038 27799 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d2e375a1ba543f0ae765fab7a56ca28 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d2e375a1ba543f0ae765fab7a56ca28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d2e375a1ba543f0ae765fab7a56ca28" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 45521 } } }
I20260808 11:37:38.697284 27799 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d2e375a1ba543f0ae765fab7a56ca28 [sys.catalog]: This master's current role is: LEADER
I20260808 11:37:38.700970 27813 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:37:38.705310 27813 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:37:38.713392 27813 catalog_manager.cc:1347] Generated new cluster ID: 76b68baeb7a34f2b9b0641b051397508
I20260808 11:37:38.713505 27813 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:37:38.730746 27813 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:37:38.731928 27813 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:37:38.739413 27813 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d2e375a1ba543f0ae765fab7a56ca28: Generated new TSK 0
I20260808 11:37:38.740162 27813 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab.
Entry for principal kudu/127.24.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab.
I20260808 11:37:38.795885 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.65:0
--local_ip_for_outbound_sockets=127.24.76.65
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:45521
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:35601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:37:39.120922 27831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:37:39.121173 27831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:37:39.121309 27831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:37:39.128350 27831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:37:39.128444 27831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:37:39.128559 27831 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 11:37:39.128620 27831 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 11:37:39.128711 27831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.65
I20260808 11:37:39.138028 27831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:35601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.76.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.76.126:45521
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.65
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:37:39.139164 27831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:37:39.140432 27831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:37:39.153077 27839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:39.153116 27840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:39.153146 27843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:37:39.154034 27831 server_base.cc:1034] running on GCE node
I20260808 11:37:39.154630 27831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:37:39.155464 27831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:37:39.156674 27831 hybrid_clock.cc:648] HybridClock initialized: now 1786189059156634 us; error 28 us; skew 500 ppm
I20260808 11:37:39.156967 27831 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:37:39 dist-test-slave-tc2s krb5kdc[27653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189059, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:37:39.159993 27831 init.cc:377] Logged in from keytab as kudu/127.24.76.65@KRBTEST.COM (short username kudu)
I20260808 11:37:39.161497 27831 webserver.cc:460] Webserver started at http://127.24.76.65:38379/ using document root <none> and password file <none>
I20260808 11:37:39.162182 27831 fs_manager.cc:359] Metadata directory not provided
I20260808 11:37:39.162285 27831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:37:39.162557 27831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:37:39.165279 27831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-0/data/instance:
uuid: "3e8cee1d98e04a58a38583a87cd4acf5"
format_stamp: "Formatted at 2026-08-08 11:37:39 on dist-test-slave-tc2s"
server_key: "5fe31737339b60a3febc42d6e7f42c3c"
server_key_iv: "e4772e635c9c1747ec1f2654949d366b"
server_key_version: "encryptionkey@0"
I20260808 11:37:39.166031 27831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance:
uuid: "3e8cee1d98e04a58a38583a87cd4acf5"
format_stamp: "Formatted at 2026-08-08 11:37:39 on dist-test-slave-tc2s"
server_key: "5fe31737339b60a3febc42d6e7f42c3c"
server_key_iv: "e4772e635c9c1747ec1f2654949d366b"
server_key_version: "encryptionkey@0"
I20260808 11:37:39.170822 27831 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 11:37:39.174057 27854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:39.175287 27831 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 11:37:39.175428 27831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-0/wal
uuid: "3e8cee1d98e04a58a38583a87cd4acf5"
format_stamp: "Formatted at 2026-08-08 11:37:39 on dist-test-slave-tc2s"
server_key: "5fe31737339b60a3febc42d6e7f42c3c"
server_key_iv: "e4772e635c9c1747ec1f2654949d366b"
server_key_version: "encryptionkey@0"
I20260808 11:37:39.175608 27831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:37:39.219302 27831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:37:39.223246 27831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:37:39.223534 27831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:37:39.226187 27831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:37:39.226290 27831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:39.226406 27831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:37:39.226496 27831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 11:37:39 dist-test-slave-tc2s krb5kdc[27653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189059, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:37:39.249003 27869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:39.226699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:45133 (local address 127.24.76.126:45521)
0808 11:37:39.227164 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:39.227170 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:39.228262 (+ 1092us) server_negotiation.cc:407] Connection header received
0808 11:37:39.230978 (+ 2716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:39.230982 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:39.231049 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:39.231139 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:39.233462 (+ 2323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:39.234401 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:39.236915 (+ 2514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:39.237347 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:39.237483 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:39.242263 (+ 4780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:39.242269 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:39.242274 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:39.242319 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:39.245478 (+ 3159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:39.246471 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:39.246477 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:39.246481 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:39.246634 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:39.247170 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:39.247176 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:39.247180 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:39.247571 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:39.247735 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:39.248350 (+ 615us) server_negotiation.cc:299] Negotiation successful
0808 11:37:39.248594 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":252,"threads_started":1}
I20260808 11:37:39.249681 27870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:39.227273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:45521 (local address 127.24.76.65:45133)
0808 11:37:39.228083 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:39.228137 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:39.230676 (+ 2539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:39.231413 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:39.231427 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:39.232248 (+ 821us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:37:39.233243 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:39.233277 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:39.234587 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:39.234593 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:39.236775 (+ 2182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:39.236790 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:39.237641 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:39.237650 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:39.238041 (+ 391us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:39.238844 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:37:39.238880 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:37:39.241932 (+ 3052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:37:39.245662 (+ 3730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:39.245674 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:39.245689 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:39.246299 (+ 610us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:39.246733 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:39.246738 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:39.246742 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:39.247019 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:39.247702 (+ 683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:37:39.247708 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:37:39.248020 (+ 312us) client_negotiation.cc:769] Sending connection context
0808 11:37:39.248233 (+ 213us) client_negotiation.cc:240] Negotiation successful
0808 11:37:39.248591 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":307,"threads_started":1}
I20260808 11:37:39.269922 27869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:39.255818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:53357 (local address 127.24.76.126:45521)
0808 11:37:39.255969 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:39.255974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:39.256583 (+ 609us) server_negotiation.cc:407] Connection header received
0808 11:37:39.256740 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:39.256745 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:39.256806 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:39.256885 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:39.257804 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:39.258695 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:39.260612 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:39.261248 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:39.261378 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:39.263911 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:39.263918 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:39.263922 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:39.263962 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:39.266867 (+ 2905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:39.267790 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:39.267796 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:39.267800 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:39.267971 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:39.268453 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:39.268457 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:39.268460 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:39.268795 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:39.268945 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:39.269346 (+ 401us) server_negotiation.cc:299] Negotiation successful
0808 11:37:39.269560 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 11:37:39.270100 27993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:39.255922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:45521 (local address 127.24.76.65:53357)
0808 11:37:39.256489 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:39.256504 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:39.256630 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:39.256921 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:39.256926 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:39.257390 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:37:39.257671 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:39.257680 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:39.258868 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:39.258874 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:39.260443 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:39.260471 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:39.261629 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:39.261635 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:39.261798 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:39.262450 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:37:39.262474 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:37:39.263712 (+ 1238us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:37:39.266999 (+ 3287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:39.267005 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:39.267010 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:39.267611 (+ 601us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:39.268127 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:39.268132 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:39.268135 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:39.268305 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:39.268911 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:37:39.268915 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:37:39.269075 (+ 160us) client_negotiation.cc:769] Sending connection context
0808 11:37:39.269221 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 11:37:39.269588 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":310,"threads_started":1}
I20260808 11:37:39.274598 27831 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.65:34007
I20260808 11:37:39.274662 28054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.65:34007 every 8 connection(s)
I20260808 11:37:39.275830 27831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
I20260808 11:37:39.277745 28056 heartbeater.cc:347] Connected to a master server at 127.24.76.126:45521
I20260808 11:37:39.278005 28056 heartbeater.cc:464] Registering TS with master...
I20260808 11:37:39.278647 28056 heartbeater.cc:511] Master 127.24.76.126:45521 requested a full tablet report, sending...
I20260808 11:37:39.280627 27729 ts_manager.cc:194] Registered new tserver with Master: 3e8cee1d98e04a58a38583a87cd4acf5 (127.24.76.65:34007)
I20260808 11:37:39.282102 27729 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.65@KRBTEST.COM'} at 127.24.76.65:45133
I20260808 11:37:39.285545 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 27831
I20260808 11:37:39.285737 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance
I20260808 11:37:39.286391 24881 external_mini_cluster.cc:1428] Setting key 75c93d1d19b14a89d49668fccdde0616
WARNING: no policy specified for kudu/127.24.76.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab.
Entry for principal kudu/127.24.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab.
I20260808 11:37:39.331254 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.66:0
--local_ip_for_outbound_sockets=127.24.76.66
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:45521
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:35601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:37:39.654294 28065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:37:39.654552 28065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:37:39.654661 28065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:37:39.661686 28065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:37:39.661784 28065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:37:39.661923 28065 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 11:37:39.662002 28065 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 11:37:39.662104 28065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.66
I20260808 11:37:39.671917 28065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:35601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.76.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.76.126:45521
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.66
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:37:39.673053 28065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:37:39.674321 28065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:37:39.686474 28073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:39.686436 28077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:39.686400 28074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:37:39.687698 28065 server_base.cc:1034] running on GCE node
I20260808 11:37:39.688283 28065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:37:39.689126 28065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:37:39.690310 28065 hybrid_clock.cc:648] HybridClock initialized: now 1786189059690257 us; error 47 us; skew 500 ppm
I20260808 11:37:39.690598 28065 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:37:39 dist-test-slave-tc2s krb5kdc[27653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189059, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:37:39.693529 28065 init.cc:377] Logged in from keytab as kudu/127.24.76.66@KRBTEST.COM (short username kudu)
I20260808 11:37:39.694871 28065 webserver.cc:460] Webserver started at http://127.24.76.66:45211/ using document root <none> and password file <none>
I20260808 11:37:39.695534 28065 fs_manager.cc:359] Metadata directory not provided
I20260808 11:37:39.695652 28065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:37:39.695940 28065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:37:39.698835 28065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-1/data/instance:
uuid: "30ff76bb491e45d88aafce3ec1eb4699"
format_stamp: "Formatted at 2026-08-08 11:37:39 on dist-test-slave-tc2s"
server_key: "6f5b99caf22c63bd4be4529b2a6125c5"
server_key_iv: "c64765ed3bd368d55c3229af02f0986e"
server_key_version: "encryptionkey@0"
I20260808 11:37:39.699651 28065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance:
uuid: "30ff76bb491e45d88aafce3ec1eb4699"
format_stamp: "Formatted at 2026-08-08 11:37:39 on dist-test-slave-tc2s"
server_key: "6f5b99caf22c63bd4be4529b2a6125c5"
server_key_iv: "c64765ed3bd368d55c3229af02f0986e"
server_key_version: "encryptionkey@0"
I20260808 11:37:39.704487 28065 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 11:37:39.707706 28088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:39.708879 28065 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 11:37:39.709036 28065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-1/wal
uuid: "30ff76bb491e45d88aafce3ec1eb4699"
format_stamp: "Formatted at 2026-08-08 11:37:39 on dist-test-slave-tc2s"
server_key: "6f5b99caf22c63bd4be4529b2a6125c5"
server_key_iv: "c64765ed3bd368d55c3229af02f0986e"
server_key_version: "encryptionkey@0"
I20260808 11:37:39.709211 28065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:37:39.720665 28065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:37:39.724583 28065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:37:39.724864 28065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:37:39.727341 28065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:37:39.727443 28065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:39.727571 28065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:37:39.727645 28065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 11:37:39 dist-test-slave-tc2s krb5kdc[27653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189059, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:37:39.748013 27869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:39.727661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:55551 (local address 127.24.76.126:45521)
0808 11:37:39.727841 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:39.727846 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:39.729129 (+ 1283us) server_negotiation.cc:407] Connection header received
0808 11:37:39.731158 (+ 2029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:39.731164 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:39.731259 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:39.731387 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:39.733380 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:39.734394 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:39.736719 (+ 2325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:39.737369 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:39.737525 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:39.741709 (+ 4184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:39.741717 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:39.741721 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:39.741766 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:39.744355 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:39.745372 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:39.745378 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:39.745381 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:39.745537 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:39.746122 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:39.746129 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:39.746133 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:39.746543 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:39.746715 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:39.747348 (+ 633us) server_negotiation.cc:299] Negotiation successful
0808 11:37:39.747612 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 11:37:39.748703 28104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:39.728133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:45521 (local address 127.24.76.66:55551)
0808 11:37:39.728927 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:39.728991 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:39.730848 (+ 1857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:39.731626 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:39.731636 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:39.732397 (+ 761us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:37:39.733118 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:39.733147 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:39.734567 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:39.734574 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:39.736553 (+ 1979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:39.736569 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:39.737685 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:39.737698 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:39.737928 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:39.738579 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:37:39.738600 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:37:39.741463 (+ 2863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:37:39.744492 (+ 3029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:39.744501 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:39.744518 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:39.745238 (+ 720us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:39.745686 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:39.745691 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:39.745695 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:39.745979 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:39.746674 (+ 695us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:37:39.746680 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:37:39.746993 (+ 313us) client_negotiation.cc:769] Sending connection context
0808 11:37:39.747203 (+ 210us) client_negotiation.cc:240] Negotiation successful
0808 11:37:39.747541 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":339,"threads_started":1}
I20260808 11:37:39.766235 27869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:39.754241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:51623 (local address 127.24.76.126:45521)
0808 11:37:39.754408 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:39.754412 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:39.755091 (+ 679us) server_negotiation.cc:407] Connection header received
0808 11:37:39.755238 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:39.755243 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:39.755302 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:39.755376 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:39.756231 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:39.756912 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:39.758301 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:39.758696 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:39.758801 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:39.760442 (+ 1641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:39.760447 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:39.760452 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:39.760487 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:39.763133 (+ 2646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:39.764056 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:39.764062 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:39.764065 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:39.764220 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:39.764648 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:39.764653 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:39.764657 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:39.765026 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:39.765172 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:39.765693 (+ 521us) server_negotiation.cc:299] Negotiation successful
0808 11:37:39.765917 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 11:37:39.766499 28195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:39.754392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:45521 (local address 127.24.76.66:51623)
0808 11:37:39.754998 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:39.755013 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:39.755130 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:39.755413 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:39.755417 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:39.755819 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:37:39.756101 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:39.756110 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:39.757047 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:39.757053 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:39.758089 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:39.758101 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:39.758872 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:39.758878 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:39.759044 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:39.759551 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:37:39.759572 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:37:39.760254 (+ 682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:37:39.763295 (+ 3041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:39.763300 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:39.763307 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:39.763932 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:39.764330 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:39.764339 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:39.764343 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:39.764502 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:39.765173 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:37:39.765179 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:37:39.765408 (+ 229us) client_negotiation.cc:769] Sending connection context
0808 11:37:39.765593 (+ 185us) client_negotiation.cc:240] Negotiation successful
0808 11:37:39.765975 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":332,"threads_started":1}
I20260808 11:37:39.779251 28065 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.66:37571
I20260808 11:37:39.779333 28294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.66:37571 every 8 connection(s)
I20260808 11:37:39.780465 28065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
I20260808 11:37:39.782303 28296 heartbeater.cc:347] Connected to a master server at 127.24.76.126:45521
I20260808 11:37:39.782521 28296 heartbeater.cc:464] Registering TS with master...
I20260808 11:37:39.783114 28296 heartbeater.cc:511] Master 127.24.76.126:45521 requested a full tablet report, sending...
I20260808 11:37:39.784325 27729 ts_manager.cc:194] Registered new tserver with Master: 30ff76bb491e45d88aafce3ec1eb4699 (127.24.76.66:37571)
I20260808 11:37:39.785077 27729 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.66@KRBTEST.COM'} at 127.24.76.66:55551
I20260808 11:37:39.788899 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 28065
I20260808 11:37:39.789059 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance
I20260808 11:37:39.789476 24881 external_mini_cluster.cc:1428] Setting key 4571b3e0d806499761ce78b1004b0fef
WARNING: no policy specified for kudu/127.24.76.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab.
Entry for principal kudu/127.24.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab.
I20260808 11:37:39.833166 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.67:0
--local_ip_for_outbound_sockets=127.24.76.67
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:45521
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:35601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:37:40.151121 28307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:37:40.151388 28307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:37:40.151532 28307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:37:40.158560 28307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:37:40.158663 28307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:37:40.158787 28307 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 11:37:40.158854 28307 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 11:37:40.158952 28307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.67
I20260808 11:37:40.168192 28307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:35601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.76.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.76.126:45521
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.67
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:37:40.169389 28307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:37:40.170643 28307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:37:40.183178 28313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:40.183137 28314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:40.183213 28316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:37:40.183794 28307 server_base.cc:1034] running on GCE node
I20260808 11:37:40.184401 28307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:37:40.185241 28307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:37:40.186436 28307 hybrid_clock.cc:648] HybridClock initialized: now 1786189060186387 us; error 41 us; skew 500 ppm
I20260808 11:37:40.186723 28307 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:37:40 dist-test-slave-tc2s krb5kdc[27653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189060, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:37:40.189711 28307 init.cc:377] Logged in from keytab as kudu/127.24.76.67@KRBTEST.COM (short username kudu)
I20260808 11:37:40.191185 28307 webserver.cc:460] Webserver started at http://127.24.76.67:40971/ using document root <none> and password file <none>
I20260808 11:37:40.191777 28307 fs_manager.cc:359] Metadata directory not provided
I20260808 11:37:40.191876 28307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:37:40.192142 28307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:37:40.195082 28307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-2/data/instance:
uuid: "7f5383ff5d894ed3b55c11b86af8af37"
format_stamp: "Formatted at 2026-08-08 11:37:40 on dist-test-slave-tc2s"
server_key: "98c578ad0bc3e66fcc08ac86a96b3dcc"
server_key_iv: "e6cb6c52cf07a7bfcf71e7759925f49e"
server_key_version: "encryptionkey@0"
I20260808 11:37:40.195801 28307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance:
uuid: "7f5383ff5d894ed3b55c11b86af8af37"
format_stamp: "Formatted at 2026-08-08 11:37:40 on dist-test-slave-tc2s"
server_key: "98c578ad0bc3e66fcc08ac86a96b3dcc"
server_key_iv: "e6cb6c52cf07a7bfcf71e7759925f49e"
server_key_version: "encryptionkey@0"
I20260808 11:37:40.200801 28307 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 11:37:40.204010 28329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:40.205159 28307 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 11:37:40.205296 28307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-2/wal
uuid: "7f5383ff5d894ed3b55c11b86af8af37"
format_stamp: "Formatted at 2026-08-08 11:37:40 on dist-test-slave-tc2s"
server_key: "98c578ad0bc3e66fcc08ac86a96b3dcc"
server_key_iv: "e6cb6c52cf07a7bfcf71e7759925f49e"
server_key_version: "encryptionkey@0"
I20260808 11:37:40.205475 28307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:37:40.224762 28307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:37:40.228677 28307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:37:40.229019 28307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:37:40.231542 28307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:37:40.231642 28307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:40.231750 28307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:37:40.231841 28307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 11:37:40 dist-test-slave-tc2s krb5kdc[27653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189060, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:37:40.253635 27869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:40.232124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:38043 (local address 127.24.76.126:45521)
0808 11:37:40.232316 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:40.232322 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:40.233590 (+ 1268us) server_negotiation.cc:407] Connection header received
0808 11:37:40.235626 (+ 2036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:40.235633 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:40.235726 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:40.235843 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:40.237852 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.238823 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.241287 (+ 2464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.241834 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.241973 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:40.246243 (+ 4270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:40.246251 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:40.246256 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:40.246302 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:40.249948 (+ 3646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:40.250935 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:40.250940 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:40.250944 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:40.251104 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:40.251644 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:40.251649 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:40.251653 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:40.252006 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:40.252165 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:40.253053 (+ 888us) server_negotiation.cc:299] Negotiation successful
0808 11:37:40.253288 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 11:37:40.254616 28343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:40.232579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:45521 (local address 127.24.76.67:38043)
0808 11:37:40.233413 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:40.233459 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:40.235329 (+ 1870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:40.236122 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:40.236130 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:40.236803 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:37:40.237600 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:40.237629 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.239022 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.239027 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:40.241151 (+ 2124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:40.241167 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.242101 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.242113 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:40.242364 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:40.243046 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:37:40.243080 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:37:40.246003 (+ 2923us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:37:40.250105 (+ 4102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:40.250115 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:40.250129 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:40.250762 (+ 633us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:40.251246 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:40.251252 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:40.251256 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:40.251540 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:40.252275 (+ 735us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:37:40.252283 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:37:40.252672 (+ 389us) client_negotiation.cc:769] Sending connection context
0808 11:37:40.252917 (+ 245us) client_negotiation.cc:240] Negotiation successful
0808 11:37:40.253400 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":369,"threads_started":1}
I20260808 11:37:40.274899 27869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:40.259733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:38435 (local address 127.24.76.126:45521)
0808 11:37:40.259877 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:40.259881 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:40.260953 (+ 1072us) server_negotiation.cc:407] Connection header received
0808 11:37:40.261116 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:40.261122 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:40.261206 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:40.261299 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:40.262764 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.263637 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.265783 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.266203 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.266294 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:40.268514 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:40.268521 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:40.268526 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:40.268564 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:40.271746 (+ 3182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:40.272660 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:40.272664 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:40.272667 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:40.272823 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:40.273306 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:40.273311 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:40.273315 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:40.273645 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:40.273793 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:40.274341 (+ 548us) server_negotiation.cc:299] Negotiation successful
0808 11:37:40.274563 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 11:37:40.275085 28471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:40.260309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:45521 (local address 127.24.76.67:38435)
0808 11:37:40.260849 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:40.260868 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:40.261008 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:40.261361 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:40.261366 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:40.262126 (+ 760us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:37:40.262596 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:40.262617 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.263782 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.263787 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:40.265647 (+ 1860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:40.265660 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.266401 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.266408 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:40.266591 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:40.267236 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:37:40.267263 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:37:40.268263 (+ 1000us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:37:40.271908 (+ 3645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:40.271913 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:40.271917 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:40.272511 (+ 594us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:40.272976 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:40.272984 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:40.272988 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:40.273157 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:40.273782 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:37:40.273787 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:37:40.274032 (+ 245us) client_negotiation.cc:769] Sending connection context
0808 11:37:40.274218 (+ 186us) client_negotiation.cc:240] Negotiation successful
0808 11:37:40.274591 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":282,"threads_started":1}
W20260808 11:37:40.275754 28048 messenger.cc:188] Invalid argument: socket address is ambiguous: no data on pending connections for acceptor pool at 127.24.76.65:34007
I20260808 11:37:40.282724 28307 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.67:39691
I20260808 11:37:40.282780 28543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.67:39691 every 8 connection(s)
I20260808 11:37:40.283857 28307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
I20260808 11:37:40.284452 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 28307
I20260808 11:37:40.284617 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance
I20260808 11:37:40.284618 28056 heartbeater.cc:503] Master 127.24.76.126:45521 was elected leader, sending a full tablet report...
I20260808 11:37:40.285096 24881 external_mini_cluster.cc:1428] Setting key b2ef528721e9cc45e62286ac834117e6
I20260808 11:37:40.286267 28545 heartbeater.cc:347] Connected to a master server at 127.24.76.126:45521
I20260808 11:37:40.286556 28545 heartbeater.cc:464] Registering TS with master...
I20260808 11:37:40.287321 28545 heartbeater.cc:511] Master 127.24.76.126:45521 requested a full tablet report, sending...
I20260808 11:37:40.288623 27724 ts_manager.cc:194] Registered new tserver with Master: 7f5383ff5d894ed3b55c11b86af8af37 (127.24.76.67:39691)
I20260808 11:37:40.289429 27724 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.67@KRBTEST.COM'} at 127.24.76.67:38043
I20260808 11:37:40.300164 24881 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 11:37:40 dist-test-slave-tc2s krb5kdc[27653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 11:37:40 dist-test-slave-tc2s krb5kdc[27653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:37:40.348631 27869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:40.336186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60416 (local address 127.24.76.126:45521)
0808 11:37:40.336357 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:40.336362 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:40.336703 (+ 341us) server_negotiation.cc:407] Connection header received
0808 11:37:40.336877 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:40.336881 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:40.336964 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:40.337045 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:40.338450 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.339201 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.340324 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.340729 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.340816 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:40.344361 (+ 3545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:40.344365 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:40.344369 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:40.344398 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:40.346674 (+ 2276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:40.347193 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:40.347197 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:40.347199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:40.347310 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:40.347579 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:40.347582 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:40.347584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:40.347828 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:40.348021 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:40.348206 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 11:37:40.348364 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 11:37:40.352561 27724 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60416:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:37:40.359692 27724 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:37:40.394733 28570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:40.370812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:34007 (local address 127.0.0.1:36300)
0808 11:37:40.380792 (+ 9980us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:40.380831 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:40.381014 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:40.386987 (+ 5973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:40.386994 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:40.387033 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:37:40.387431 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:40.387445 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.390382 (+ 2937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.390390 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:40.392204 (+ 1814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:40.392218 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.393759 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.393765 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:40.393994 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:40.394048 (+ 54us) client_negotiation.cc:769] Sending connection context
0808 11:37:40.394179 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 11:37:40.394359 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9721,"mutex_wait_us":2,"thread_start_us":18014,"threads_started":1}
I20260808 11:37:40.395344 28571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:40.371096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36300 (local address 127.24.76.65:34007)
0808 11:37:40.386330 (+ 15234us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:40.386341 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:40.386373 (+ 32us) server_negotiation.cc:407] Connection header received
0808 11:37:40.386490 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:40.386498 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:40.386755 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:40.386963 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:37:40.387768 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.390245 (+ 2477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.392417 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.393549 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.393670 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:40.393911 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:40.394962 (+ 1051us) server_negotiation.cc:299] Negotiation successful
0808 11:37:40.395128 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15050,"thread_start_us":16436,"threads_started":1}
I20260808 11:37:40.396968 28574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:40.381463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60272 (local address 127.24.76.66:37571)
0808 11:37:40.388849 (+ 7386us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:40.388867 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:40.389371 (+ 504us) server_negotiation.cc:407] Connection header received
0808 11:37:40.389483 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:40.389490 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:40.389695 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:40.389868 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:37:40.390520 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.391901 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.394466 (+ 2565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.395828 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.395952 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:40.396147 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:40.396643 (+ 496us) server_negotiation.cc:299] Negotiation successful
0808 11:37:40.396768 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7135,"thread_start_us":7006,"threads_started":1}
I20260808 11:37:40.397065 28566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:40.386995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:37571 (local address 127.0.0.1:60272)
0808 11:37:40.389113 (+ 2118us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:40.389129 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:40.389252 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:40.390073 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:40.390077 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:40.390094 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:37:40.390357 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:40.390367 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.392081 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.392087 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:40.394289 (+ 2202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:40.394304 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.396305 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.396312 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:40.396467 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:40.396505 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 11:37:40.396585 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 11:37:40.396697 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1997,"mutex_wait_us":1,"thread_start_us":886,"threads_started":1}
I20260808 11:37:40.397634 28572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:40.371512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47926 (local address 127.24.76.67:39691)
0808 11:37:40.387654 (+ 16142us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:40.387661 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:40.388246 (+ 585us) server_negotiation.cc:407] Connection header received
0808 11:37:40.388427 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:40.388435 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:40.388666 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:40.388844 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:37:40.389701 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.391264 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.393777 (+ 2513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.395181 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.395281 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:40.395502 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:40.397144 (+ 1642us) server_negotiation.cc:299] Negotiation successful
0808 11:37:40.397306 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15983,"thread_start_us":15810,"threads_started":1}
I20260808 11:37:40.397653 28575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:40.371042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:39691 (local address 127.0.0.1:47926)
0808 11:37:40.388136 (+ 17094us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:40.388156 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:40.388288 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:40.388961 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:40.388965 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:40.388995 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:37:40.389458 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:40.389470 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.391422 (+ 1952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.391427 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:40.393585 (+ 2158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:40.393601 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.395692 (+ 2091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.395698 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:40.395870 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:40.395903 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 11:37:40.395986 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 11:37:40.396107 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16972,"mutex_wait_us":28,"thread_start_us":9352,"threads_started":1}
I20260808 11:37:40.399966 27944 tablet_service.cc:1467] Processing CreateTablet for tablet 042f264de9f0422b89fe3176919724d2 (DEFAULT_TABLE table=test-table [id=abe5d416d2d7428aa0ca3d0387a4db1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:37:40.400256 28174 tablet_service.cc:1467] Processing CreateTablet for tablet 042f264de9f0422b89fe3176919724d2 (DEFAULT_TABLE table=test-table [id=abe5d416d2d7428aa0ca3d0387a4db1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:37:40.401413 28174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 042f264de9f0422b89fe3176919724d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:37:40.401439 27944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 042f264de9f0422b89fe3176919724d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:37:40.400964 28409 tablet_service.cc:1467] Processing CreateTablet for tablet 042f264de9f0422b89fe3176919724d2 (DEFAULT_TABLE table=test-table [id=abe5d416d2d7428aa0ca3d0387a4db1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:37:40.402231 28409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 042f264de9f0422b89fe3176919724d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:37:40.412400 28576 tablet_bootstrap.cc:492] T 042f264de9f0422b89fe3176919724d2 P 30ff76bb491e45d88aafce3ec1eb4699: Bootstrap starting.
I20260808 11:37:40.415967 28576 tablet_bootstrap.cc:654] T 042f264de9f0422b89fe3176919724d2 P 30ff76bb491e45d88aafce3ec1eb4699: Neither blocks nor log segments found. Creating new log.
I20260808 11:37:40.415989 28577 tablet_bootstrap.cc:492] T 042f264de9f0422b89fe3176919724d2 P 3e8cee1d98e04a58a38583a87cd4acf5: Bootstrap starting.
I20260808 11:37:40.417311 28576 log.cc:824] T 042f264de9f0422b89fe3176919724d2 P 30ff76bb491e45d88aafce3ec1eb4699: Log is configured to *not* fsync() on all Append() calls
I20260808 11:37:40.417393 28580 tablet_bootstrap.cc:492] T 042f264de9f0422b89fe3176919724d2 P 7f5383ff5d894ed3b55c11b86af8af37: Bootstrap starting.
I20260808 11:37:40.419267 28577 tablet_bootstrap.cc:654] T 042f264de9f0422b89fe3176919724d2 P 3e8cee1d98e04a58a38583a87cd4acf5: Neither blocks nor log segments found. Creating new log.
I20260808 11:37:40.420231 28580 tablet_bootstrap.cc:654] T 042f264de9f0422b89fe3176919724d2 P 7f5383ff5d894ed3b55c11b86af8af37: Neither blocks nor log segments found. Creating new log.
I20260808 11:37:40.420466 28577 log.cc:824] T 042f264de9f0422b89fe3176919724d2 P 3e8cee1d98e04a58a38583a87cd4acf5: Log is configured to *not* fsync() on all Append() calls
I20260808 11:37:40.420697 28576 tablet_bootstrap.cc:492] T 042f264de9f0422b89fe3176919724d2 P 30ff76bb491e45d88aafce3ec1eb4699: No bootstrap required, opened a new log
I20260808 11:37:40.420960 28576 ts_tablet_manager.cc:1397] T 042f264de9f0422b89fe3176919724d2 P 30ff76bb491e45d88aafce3ec1eb4699: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260808 11:37:40.421250 28580 log.cc:824] T 042f264de9f0422b89fe3176919724d2 P 7f5383ff5d894ed3b55c11b86af8af37: Log is configured to *not* fsync() on all Append() calls
I20260808 11:37:40.423427 28577 tablet_bootstrap.cc:492] T 042f264de9f0422b89fe3176919724d2 P 3e8cee1d98e04a58a38583a87cd4acf5: No bootstrap required, opened a new log
I20260808 11:37:40.423698 28577 ts_tablet_manager.cc:1397] T 042f264de9f0422b89fe3176919724d2 P 3e8cee1d98e04a58a38583a87cd4acf5: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260808 11:37:40.424754 28580 tablet_bootstrap.cc:492] T 042f264de9f0422b89fe3176919724d2 P 7f5383ff5d894ed3b55c11b86af8af37: No bootstrap required, opened a new log
I20260808 11:37:40.425014 28580 ts_tablet_manager.cc:1397] T 042f264de9f0422b89fe3176919724d2 P 7f5383ff5d894ed3b55c11b86af8af37: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260808 11:37:40.428520 28576 raft_consensus.cc:348] T 042f264de9f0422b89fe3176919724d2 P 30ff76bb491e45d88aafce3ec1eb4699 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } }
I20260808 11:37:40.428834 28576 raft_consensus.cc:374] T 042f264de9f0422b89fe3176919724d2 P 30ff76bb491e45d88aafce3ec1eb4699 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:37:40.428956 28576 raft_consensus.cc:729] T 042f264de9f0422b89fe3176919724d2 P 30ff76bb491e45d88aafce3ec1eb4699 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30ff76bb491e45d88aafce3ec1eb4699, State: Initialized, Role: FOLLOWER
I20260808 11:37:40.429494 28576 consensus_queue.cc:260] T 042f264de9f0422b89fe3176919724d2 P 30ff76bb491e45d88aafce3ec1eb4699 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } }
I20260808 11:37:40.431633 28296 heartbeater.cc:503] Master 127.24.76.126:45521 was elected leader, sending a full tablet report...
I20260808 11:37:40.432082 28576 ts_tablet_manager.cc:1428] T 042f264de9f0422b89fe3176919724d2 P 30ff76bb491e45d88aafce3ec1eb4699: Time spent starting tablet: real 0.011s user 0.004s sys 0.006s
I20260808 11:37:40.433609 28577 raft_consensus.cc:348] T 042f264de9f0422b89fe3176919724d2 P 3e8cee1d98e04a58a38583a87cd4acf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } }
I20260808 11:37:40.434019 28577 raft_consensus.cc:374] T 042f264de9f0422b89fe3176919724d2 P 3e8cee1d98e04a58a38583a87cd4acf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:37:40.434111 28577 raft_consensus.cc:729] T 042f264de9f0422b89fe3176919724d2 P 3e8cee1d98e04a58a38583a87cd4acf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8cee1d98e04a58a38583a87cd4acf5, State: Initialized, Role: FOLLOWER
I20260808 11:37:40.434679 28577 consensus_queue.cc:260] T 042f264de9f0422b89fe3176919724d2 P 3e8cee1d98e04a58a38583a87cd4acf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } }
I20260808 11:37:40.435693 28580 raft_consensus.cc:348] T 042f264de9f0422b89fe3176919724d2 P 7f5383ff5d894ed3b55c11b86af8af37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } }
I20260808 11:37:40.436102 28580 raft_consensus.cc:374] T 042f264de9f0422b89fe3176919724d2 P 7f5383ff5d894ed3b55c11b86af8af37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:37:40.436200 28580 raft_consensus.cc:729] T 042f264de9f0422b89fe3176919724d2 P 7f5383ff5d894ed3b55c11b86af8af37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f5383ff5d894ed3b55c11b86af8af37, State: Initialized, Role: FOLLOWER
I20260808 11:37:40.436748 28580 consensus_queue.cc:260] T 042f264de9f0422b89fe3176919724d2 P 7f5383ff5d894ed3b55c11b86af8af37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } }
I20260808 11:37:40.437443 28577 ts_tablet_manager.cc:1428] T 042f264de9f0422b89fe3176919724d2 P 3e8cee1d98e04a58a38583a87cd4acf5: Time spent starting tablet: real 0.014s user 0.008s sys 0.004s
I20260808 11:37:40.437949 28545 heartbeater.cc:503] Master 127.24.76.126:45521 was elected leader, sending a full tablet report...
I20260808 11:37:40.438424 28580 ts_tablet_manager.cc:1428] T 042f264de9f0422b89fe3176919724d2 P 7f5383ff5d894ed3b55c11b86af8af37: Time spent starting tablet: real 0.013s user 0.010s sys 0.004s
W20260808 11:37:40.528208 28058 tablet.cc:2369] T 042f264de9f0422b89fe3176919724d2 P 3e8cee1d98e04a58a38583a87cd4acf5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:37:40.532270 28297 tablet.cc:2369] T 042f264de9f0422b89fe3176919724d2 P 30ff76bb491e45d88aafce3ec1eb4699: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:37:40.536311 28550 tablet.cc:2369] T 042f264de9f0422b89fe3176919724d2 P 7f5383ff5d894ed3b55c11b86af8af37: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:37:40.610054 28588 raft_consensus.cc:482] T 042f264de9f0422b89fe3176919724d2 P 30ff76bb491e45d88aafce3ec1eb4699 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:37:40.610378 28588 raft_consensus.cc:504] T 042f264de9f0422b89fe3176919724d2 P 30ff76bb491e45d88aafce3ec1eb4699 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } }
I20260808 11:37:40.612087 28588 leader_election.cc:288] T 042f264de9f0422b89fe3176919724d2 P 30ff76bb491e45d88aafce3ec1eb4699 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e8cee1d98e04a58a38583a87cd4acf5 (127.24.76.65:34007), 7f5383ff5d894ed3b55c11b86af8af37 (127.24.76.67:39691)
I20260808 11:37:40.620703 28572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:40.612657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:43789 (local address 127.24.76.67:39691)
0808 11:37:40.612840 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:40.612845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:40.613422 (+ 577us) server_negotiation.cc:407] Connection header received
0808 11:37:40.613644 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:40.613650 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:40.613726 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:40.613820 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:37:40.614538 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.615634 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.618479 (+ 2845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.619664 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.619831 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:40.619895 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:40.620330 (+ 435us) server_negotiation.cc:299] Negotiation successful
0808 11:37:40.620457 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 11:37:40.620710 28571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:40.612531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:46043 (local address 127.24.76.65:34007)
0808 11:37:40.612746 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:40.612752 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:40.613507 (+ 755us) server_negotiation.cc:407] Connection header received
0808 11:37:40.613641 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:40.613646 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:40.613727 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:40.613835 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:37:40.614654 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.615674 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.618468 (+ 2794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.619663 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.619823 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:40.619895 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:40.620334 (+ 439us) server_negotiation.cc:299] Negotiation successful
0808 11:37:40.620457 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 11:37:40.620944 28593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:40.612353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:39691 (local address 127.24.76.66:43789)
0808 11:37:40.613307 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:40.613326 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:40.613490 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:40.613851 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:40.613856 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:40.613885 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:37:40.614392 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:40.614407 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.615785 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.615790 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:40.618336 (+ 2546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:40.618346 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.619904 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.619909 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:40.620111 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:40.620192 (+ 81us) client_negotiation.cc:769] Sending connection context
0808 11:37:40.620283 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 11:37:40.620483 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":812,"mutex_wait_us":197,"thread_start_us":665,"threads_started":1}
I20260808 11:37:40.621019 28592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:40.612464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:34007 (local address 127.24.76.66:46043)
0808 11:37:40.613397 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:40.613412 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:40.613517 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:40.613851 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:40.613856 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:40.613964 (+ 108us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:37:40.614505 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:40.614516 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.615786 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.615790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:40.618325 (+ 2535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:40.618341 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.619891 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.619898 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:40.620104 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:40.620192 (+ 88us) client_negotiation.cc:769] Sending connection context
0808 11:37:40.620280 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 11:37:40.620483 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":792,"mutex_wait_us":200,"spinlock_wait_cycles":64256,"thread_start_us":420,"threads_started":1}
I20260808 11:37:40.622182 27970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "042f264de9f0422b89fe3176919724d2" candidate_uuid: "30ff76bb491e45d88aafce3ec1eb4699" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" is_pre_election: true
I20260808 11:37:40.622182 28452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "042f264de9f0422b89fe3176919724d2" candidate_uuid: "30ff76bb491e45d88aafce3ec1eb4699" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f5383ff5d894ed3b55c11b86af8af37" is_pre_election: true
I20260808 11:37:40.622754 27970 raft_consensus.cc:2454] T 042f264de9f0422b89fe3176919724d2 P 3e8cee1d98e04a58a38583a87cd4acf5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30ff76bb491e45d88aafce3ec1eb4699 in term 0.
I20260808 11:37:40.622757 28452 raft_consensus.cc:2454] T 042f264de9f0422b89fe3176919724d2 P 7f5383ff5d894ed3b55c11b86af8af37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30ff76bb491e45d88aafce3ec1eb4699 in term 0.
I20260808 11:37:40.623450 28094 leader_election.cc:302] T 042f264de9f0422b89fe3176919724d2 P 30ff76bb491e45d88aafce3ec1eb4699 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30ff76bb491e45d88aafce3ec1eb4699, 3e8cee1d98e04a58a38583a87cd4acf5; no voters:
I20260808 11:37:40.623826 28588 raft_consensus.cc:2781] T 042f264de9f0422b89fe3176919724d2 P 30ff76bb491e45d88aafce3ec1eb4699 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:37:40.623976 28588 raft_consensus.cc:482] T 042f264de9f0422b89fe3176919724d2 P 30ff76bb491e45d88aafce3ec1eb4699 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:37:40.624099 28588 raft_consensus.cc:3037] T 042f264de9f0422b89fe3176919724d2 P 30ff76bb491e45d88aafce3ec1eb4699 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:37:40.626251 28588 raft_consensus.cc:504] T 042f264de9f0422b89fe3176919724d2 P 30ff76bb491e45d88aafce3ec1eb4699 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } }
I20260808 11:37:40.626828 28588 leader_election.cc:288] T 042f264de9f0422b89fe3176919724d2 P 30ff76bb491e45d88aafce3ec1eb4699 [CANDIDATE]: Term 1 election: Requested vote from peers 3e8cee1d98e04a58a38583a87cd4acf5 (127.24.76.65:34007), 7f5383ff5d894ed3b55c11b86af8af37 (127.24.76.67:39691)
I20260808 11:37:40.627203 27970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "042f264de9f0422b89fe3176919724d2" candidate_uuid: "30ff76bb491e45d88aafce3ec1eb4699" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8cee1d98e04a58a38583a87cd4acf5"
I20260808 11:37:40.627302 28452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "042f264de9f0422b89fe3176919724d2" candidate_uuid: "30ff76bb491e45d88aafce3ec1eb4699" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f5383ff5d894ed3b55c11b86af8af37"
I20260808 11:37:40.627414 27970 raft_consensus.cc:3037] T 042f264de9f0422b89fe3176919724d2 P 3e8cee1d98e04a58a38583a87cd4acf5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:37:40.627485 28452 raft_consensus.cc:3037] T 042f264de9f0422b89fe3176919724d2 P 7f5383ff5d894ed3b55c11b86af8af37 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:37:40.630636 28452 raft_consensus.cc:2454] T 042f264de9f0422b89fe3176919724d2 P 7f5383ff5d894ed3b55c11b86af8af37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30ff76bb491e45d88aafce3ec1eb4699 in term 1.
I20260808 11:37:40.630636 27970 raft_consensus.cc:2454] T 042f264de9f0422b89fe3176919724d2 P 3e8cee1d98e04a58a38583a87cd4acf5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30ff76bb491e45d88aafce3ec1eb4699 in term 1.
I20260808 11:37:40.631024 28089 leader_election.cc:302] T 042f264de9f0422b89fe3176919724d2 P 30ff76bb491e45d88aafce3ec1eb4699 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30ff76bb491e45d88aafce3ec1eb4699, 7f5383ff5d894ed3b55c11b86af8af37; no voters:
I20260808 11:37:40.631310 28588 raft_consensus.cc:2781] T 042f264de9f0422b89fe3176919724d2 P 30ff76bb491e45d88aafce3ec1eb4699 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:37:40.631685 28588 raft_consensus.cc:686] T 042f264de9f0422b89fe3176919724d2 P 30ff76bb491e45d88aafce3ec1eb4699 [term 1 LEADER]: Becoming Leader. State: Replica: 30ff76bb491e45d88aafce3ec1eb4699, State: Running, Role: LEADER
I20260808 11:37:40.632108 28588 consensus_queue.cc:237] T 042f264de9f0422b89fe3176919724d2 P 30ff76bb491e45d88aafce3ec1eb4699 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } }
I20260808 11:37:40.636643 27729 catalog_manager.cc:5495] T 042f264de9f0422b89fe3176919724d2 P 30ff76bb491e45d88aafce3ec1eb4699 reported cstate change: term changed from 0 to 1, leader changed from <none> to 30ff76bb491e45d88aafce3ec1eb4699 (127.24.76.66). New cstate: current_term: 1 leader_uuid: "30ff76bb491e45d88aafce3ec1eb4699" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } health_report { overall_health: HEALTHY } } }
Aug 08 11:37:40 dist-test-slave-tc2s krb5kdc[27653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189058, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:37:40.720819 28609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:40.708133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:45521 (local address 127.0.0.1:60418)
0808 11:37:40.708659 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:40.708674 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:40.708804 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:40.709199 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:40.709204 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:40.710034 (+ 830us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:37:40.710313 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:40.710323 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.711341 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.711345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:40.712339 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:40.712351 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.713061 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.713067 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:40.713201 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:40.713779 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:37:40.713797 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:37:40.715983 (+ 2186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:37:40.718707 (+ 2724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:40.718718 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:40.718724 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:40.719100 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:40.719411 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:40.719414 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:40.719417 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:40.719554 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:40.720005 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:37:40.720011 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:37:40.720166 (+ 155us) client_negotiation.cc:769] Sending connection context
0808 11:37:40.720311 (+ 145us) client_negotiation.cc:240] Negotiation successful
0808 11:37:40.720509 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":263,"threads_started":1}
I20260808 11:37:40.720839 27869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:40.708061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60418 (local address 127.24.76.126:45521)
0808 11:37:40.708223 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:40.708228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:40.708751 (+ 523us) server_negotiation.cc:407] Connection header received
0808 11:37:40.708952 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:40.708956 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:40.709020 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:40.709138 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:40.710457 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.711207 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.712479 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.712881 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.712987 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:40.716161 (+ 3174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:40.716166 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:40.716169 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:40.716199 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:40.718573 (+ 2374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:40.719192 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:40.719196 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:40.719198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:40.719318 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:40.719643 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:40.719646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:40.719648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:40.719902 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:40.720006 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:40.720402 (+ 396us) server_negotiation.cc:299] Negotiation successful
0808 11:37:40.720613 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":52}
I20260808 11:37:40.725306 27729 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:60418:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 11:37:40.726341 27729 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:37:40.732636 27944 tablet_service.cc:1467] Processing CreateTablet for tablet c48b211d295c49b694f5823af68a1b6f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cc3aadb1796444d1b652f36768a07097]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 11:37:40.732918 28409 tablet_service.cc:1467] Processing CreateTablet for tablet c48b211d295c49b694f5823af68a1b6f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cc3aadb1796444d1b652f36768a07097]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 11:37:40.732858 28174 tablet_service.cc:1467] Processing CreateTablet for tablet c48b211d295c49b694f5823af68a1b6f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cc3aadb1796444d1b652f36768a07097]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 11:37:40.733577 27944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c48b211d295c49b694f5823af68a1b6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:37:40.733734 28174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c48b211d295c49b694f5823af68a1b6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:37:40.733814 28409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c48b211d295c49b694f5823af68a1b6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:37:40.738621 28577 tablet_bootstrap.cc:492] T c48b211d295c49b694f5823af68a1b6f P 3e8cee1d98e04a58a38583a87cd4acf5: Bootstrap starting.
I20260808 11:37:40.740041 28577 tablet_bootstrap.cc:654] T c48b211d295c49b694f5823af68a1b6f P 3e8cee1d98e04a58a38583a87cd4acf5: Neither blocks nor log segments found. Creating new log.
I20260808 11:37:40.740549 28576 tablet_bootstrap.cc:492] T c48b211d295c49b694f5823af68a1b6f P 30ff76bb491e45d88aafce3ec1eb4699: Bootstrap starting.
I20260808 11:37:40.740549 28580 tablet_bootstrap.cc:492] T c48b211d295c49b694f5823af68a1b6f P 7f5383ff5d894ed3b55c11b86af8af37: Bootstrap starting.
I20260808 11:37:40.742193 28580 tablet_bootstrap.cc:654] T c48b211d295c49b694f5823af68a1b6f P 7f5383ff5d894ed3b55c11b86af8af37: Neither blocks nor log segments found. Creating new log.
I20260808 11:37:40.742254 28576 tablet_bootstrap.cc:654] T c48b211d295c49b694f5823af68a1b6f P 30ff76bb491e45d88aafce3ec1eb4699: Neither blocks nor log segments found. Creating new log.
I20260808 11:37:40.747463 28577 tablet_bootstrap.cc:492] T c48b211d295c49b694f5823af68a1b6f P 3e8cee1d98e04a58a38583a87cd4acf5: No bootstrap required, opened a new log
I20260808 11:37:40.747611 28577 ts_tablet_manager.cc:1397] T c48b211d295c49b694f5823af68a1b6f P 3e8cee1d98e04a58a38583a87cd4acf5: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260808 11:37:40.748253 28576 tablet_bootstrap.cc:492] T c48b211d295c49b694f5823af68a1b6f P 30ff76bb491e45d88aafce3ec1eb4699: No bootstrap required, opened a new log
I20260808 11:37:40.748245 28577 raft_consensus.cc:348] T c48b211d295c49b694f5823af68a1b6f P 3e8cee1d98e04a58a38583a87cd4acf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } }
I20260808 11:37:40.748383 28576 ts_tablet_manager.cc:1397] T c48b211d295c49b694f5823af68a1b6f P 30ff76bb491e45d88aafce3ec1eb4699: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260808 11:37:40.748457 28577 raft_consensus.cc:374] T c48b211d295c49b694f5823af68a1b6f P 3e8cee1d98e04a58a38583a87cd4acf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:37:40.748551 28577 raft_consensus.cc:729] T c48b211d295c49b694f5823af68a1b6f P 3e8cee1d98e04a58a38583a87cd4acf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8cee1d98e04a58a38583a87cd4acf5, State: Initialized, Role: FOLLOWER
I20260808 11:37:40.748772 28577 consensus_queue.cc:260] T c48b211d295c49b694f5823af68a1b6f P 3e8cee1d98e04a58a38583a87cd4acf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } }
I20260808 11:37:40.749148 28576 raft_consensus.cc:348] T c48b211d295c49b694f5823af68a1b6f P 30ff76bb491e45d88aafce3ec1eb4699 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } }
I20260808 11:37:40.749437 28576 raft_consensus.cc:374] T c48b211d295c49b694f5823af68a1b6f P 30ff76bb491e45d88aafce3ec1eb4699 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:37:40.749264 28589 tablet_replica.cc:439] T c48b211d295c49b694f5823af68a1b6f P 3e8cee1d98e04a58a38583a87cd4acf5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } } }
I20260808 11:37:40.749571 28576 raft_consensus.cc:729] T c48b211d295c49b694f5823af68a1b6f P 30ff76bb491e45d88aafce3ec1eb4699 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30ff76bb491e45d88aafce3ec1eb4699, State: Initialized, Role: FOLLOWER
I20260808 11:37:40.749671 28589 tablet_replica.cc:442] T c48b211d295c49b694f5823af68a1b6f P 3e8cee1d98e04a58a38583a87cd4acf5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:37:40.749881 28580 tablet_bootstrap.cc:492] T c48b211d295c49b694f5823af68a1b6f P 7f5383ff5d894ed3b55c11b86af8af37: No bootstrap required, opened a new log
I20260808 11:37:40.750031 28577 ts_tablet_manager.cc:1428] T c48b211d295c49b694f5823af68a1b6f P 3e8cee1d98e04a58a38583a87cd4acf5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 11:37:40.750098 28580 ts_tablet_manager.cc:1397] T c48b211d295c49b694f5823af68a1b6f P 7f5383ff5d894ed3b55c11b86af8af37: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260808 11:37:40.750154 28576 consensus_queue.cc:260] T c48b211d295c49b694f5823af68a1b6f P 30ff76bb491e45d88aafce3ec1eb4699 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } }
I20260808 11:37:40.750571 28588 tablet_replica.cc:439] T c48b211d295c49b694f5823af68a1b6f P 30ff76bb491e45d88aafce3ec1eb4699: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } } }
I20260808 11:37:40.750830 28588 tablet_replica.cc:442] T c48b211d295c49b694f5823af68a1b6f P 30ff76bb491e45d88aafce3ec1eb4699: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:37:40.750869 28580 raft_consensus.cc:348] T c48b211d295c49b694f5823af68a1b6f P 7f5383ff5d894ed3b55c11b86af8af37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } }
I20260808 11:37:40.751183 28580 raft_consensus.cc:374] T c48b211d295c49b694f5823af68a1b6f P 7f5383ff5d894ed3b55c11b86af8af37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:37:40.751293 28580 raft_consensus.cc:729] T c48b211d295c49b694f5823af68a1b6f P 7f5383ff5d894ed3b55c11b86af8af37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f5383ff5d894ed3b55c11b86af8af37, State: Initialized, Role: FOLLOWER
I20260808 11:37:40.751516 28580 consensus_queue.cc:260] T c48b211d295c49b694f5823af68a1b6f P 7f5383ff5d894ed3b55c11b86af8af37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } }
I20260808 11:37:40.751873 28576 ts_tablet_manager.cc:1428] T c48b211d295c49b694f5823af68a1b6f P 30ff76bb491e45d88aafce3ec1eb4699: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 11:37:40.752079 28580 ts_tablet_manager.cc:1428] T c48b211d295c49b694f5823af68a1b6f P 7f5383ff5d894ed3b55c11b86af8af37: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260808 11:37:40.752281 28590 tablet_replica.cc:439] T c48b211d295c49b694f5823af68a1b6f P 7f5383ff5d894ed3b55c11b86af8af37: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } } }
I20260808 11:37:40.752738 28590 tablet_replica.cc:442] T c48b211d295c49b694f5823af68a1b6f P 7f5383ff5d894ed3b55c11b86af8af37: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:37:40.937397 28590 raft_consensus.cc:482] T c48b211d295c49b694f5823af68a1b6f P 7f5383ff5d894ed3b55c11b86af8af37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:37:40.937604 28590 raft_consensus.cc:504] T c48b211d295c49b694f5823af68a1b6f P 7f5383ff5d894ed3b55c11b86af8af37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } }
I20260808 11:37:40.939122 28590 leader_election.cc:288] T c48b211d295c49b694f5823af68a1b6f P 7f5383ff5d894ed3b55c11b86af8af37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e8cee1d98e04a58a38583a87cd4acf5 (127.24.76.65:34007), 30ff76bb491e45d88aafce3ec1eb4699 (127.24.76.66:37571)
I20260808 11:37:40.945084 28571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:40.939629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:60999 (local address 127.24.76.65:34007)
0808 11:37:40.939796 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:40.939803 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:40.940258 (+ 455us) server_negotiation.cc:407] Connection header received
0808 11:37:40.940471 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:40.940474 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:40.940539 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:40.940614 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:37:40.941067 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.941784 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.943422 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.944213 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.944385 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:40.944459 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:40.944702 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 11:37:40.944827 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 11:37:40.945185 28614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:40.939300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:34007 (local address 127.24.76.67:60999)
0808 11:37:40.940147 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:40.940163 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:40.940331 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:40.940646 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:40.940651 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:40.940677 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:37:40.940899 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:40.940908 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.941931 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.941936 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:40.943260 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:40.943276 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.944381 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.944387 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:40.944558 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:40.944583 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 11:37:40.944662 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 11:37:40.944795 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":740,"mutex_wait_us":56,"thread_start_us":645,"threads_started":1}
I20260808 11:37:40.945319 28618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:40.939532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:47101 (local address 127.24.76.66:37571)
0808 11:37:40.940035 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:40.940042 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:40.940473 (+ 431us) server_negotiation.cc:407] Connection header received
0808 11:37:40.940626 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:40.940630 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:40.940686 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:40.940766 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:37:40.941259 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.942005 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.943572 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.944369 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.944527 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:40.944599 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:40.944876 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 11:37:40.945015 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":227,"threads_started":1}
I20260808 11:37:40.945637 28617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:40.939359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:37571 (local address 127.24.76.67:47101)
0808 11:37:40.940376 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:40.940390 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:40.940508 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:40.940798 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:40.940803 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:40.940834 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:37:40.941120 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:40.941129 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.942144 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.942148 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:40.943419 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:40.943431 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.944574 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.944581 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:40.944736 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:40.944759 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 11:37:40.944835 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 11:37:40.944955 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":911,"mutex_wait_us":21,"thread_start_us":794,"threads_started":1}
I20260808 11:37:40.945869 27970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c48b211d295c49b694f5823af68a1b6f" candidate_uuid: "7f5383ff5d894ed3b55c11b86af8af37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" is_pre_election: true
I20260808 11:37:40.946168 27970 raft_consensus.cc:2454] T c48b211d295c49b694f5823af68a1b6f P 3e8cee1d98e04a58a38583a87cd4acf5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f5383ff5d894ed3b55c11b86af8af37 in term 0.
I20260808 11:37:40.946586 28334 leader_election.cc:302] T c48b211d295c49b694f5823af68a1b6f P 7f5383ff5d894ed3b55c11b86af8af37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e8cee1d98e04a58a38583a87cd4acf5, 7f5383ff5d894ed3b55c11b86af8af37; no voters:
I20260808 11:37:40.946537 28219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c48b211d295c49b694f5823af68a1b6f" candidate_uuid: "7f5383ff5d894ed3b55c11b86af8af37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ff76bb491e45d88aafce3ec1eb4699" is_pre_election: true
I20260808 11:37:40.946943 28590 raft_consensus.cc:2781] T c48b211d295c49b694f5823af68a1b6f P 7f5383ff5d894ed3b55c11b86af8af37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:37:40.946887 28219 raft_consensus.cc:2454] T c48b211d295c49b694f5823af68a1b6f P 30ff76bb491e45d88aafce3ec1eb4699 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f5383ff5d894ed3b55c11b86af8af37 in term 0.
I20260808 11:37:40.947067 28590 raft_consensus.cc:482] T c48b211d295c49b694f5823af68a1b6f P 7f5383ff5d894ed3b55c11b86af8af37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:37:40.947165 28590 raft_consensus.cc:3037] T c48b211d295c49b694f5823af68a1b6f P 7f5383ff5d894ed3b55c11b86af8af37 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:37:40.949301 28590 raft_consensus.cc:504] T c48b211d295c49b694f5823af68a1b6f P 7f5383ff5d894ed3b55c11b86af8af37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } }
I20260808 11:37:40.949808 28590 leader_election.cc:288] T c48b211d295c49b694f5823af68a1b6f P 7f5383ff5d894ed3b55c11b86af8af37 [CANDIDATE]: Term 1 election: Requested vote from peers 3e8cee1d98e04a58a38583a87cd4acf5 (127.24.76.65:34007), 30ff76bb491e45d88aafce3ec1eb4699 (127.24.76.66:37571)
I20260808 11:37:40.950248 27970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c48b211d295c49b694f5823af68a1b6f" candidate_uuid: "7f5383ff5d894ed3b55c11b86af8af37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8cee1d98e04a58a38583a87cd4acf5"
I20260808 11:37:40.950273 28219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c48b211d295c49b694f5823af68a1b6f" candidate_uuid: "7f5383ff5d894ed3b55c11b86af8af37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ff76bb491e45d88aafce3ec1eb4699"
I20260808 11:37:40.950434 28219 raft_consensus.cc:3037] T c48b211d295c49b694f5823af68a1b6f P 30ff76bb491e45d88aafce3ec1eb4699 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:37:40.950456 27970 raft_consensus.cc:3037] T c48b211d295c49b694f5823af68a1b6f P 3e8cee1d98e04a58a38583a87cd4acf5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:37:40.952616 27970 raft_consensus.cc:2454] T c48b211d295c49b694f5823af68a1b6f P 3e8cee1d98e04a58a38583a87cd4acf5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f5383ff5d894ed3b55c11b86af8af37 in term 1.
I20260808 11:37:40.952893 28219 raft_consensus.cc:2454] T c48b211d295c49b694f5823af68a1b6f P 30ff76bb491e45d88aafce3ec1eb4699 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f5383ff5d894ed3b55c11b86af8af37 in term 1.
I20260808 11:37:40.952966 28334 leader_election.cc:302] T c48b211d295c49b694f5823af68a1b6f P 7f5383ff5d894ed3b55c11b86af8af37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e8cee1d98e04a58a38583a87cd4acf5, 7f5383ff5d894ed3b55c11b86af8af37; no voters:
I20260808 11:37:40.953235 28590 raft_consensus.cc:2781] T c48b211d295c49b694f5823af68a1b6f P 7f5383ff5d894ed3b55c11b86af8af37 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:37:40.953605 28590 raft_consensus.cc:686] T c48b211d295c49b694f5823af68a1b6f P 7f5383ff5d894ed3b55c11b86af8af37 [term 1 LEADER]: Becoming Leader. State: Replica: 7f5383ff5d894ed3b55c11b86af8af37, State: Running, Role: LEADER
I20260808 11:37:40.954147 28590 consensus_queue.cc:237] T c48b211d295c49b694f5823af68a1b6f P 7f5383ff5d894ed3b55c11b86af8af37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } }
I20260808 11:37:40.957428 28619 tablet_replica.cc:439] T c48b211d295c49b694f5823af68a1b6f P 7f5383ff5d894ed3b55c11b86af8af37: TxnStatusTablet state changed. Reason: New leader 7f5383ff5d894ed3b55c11b86af8af37. Latest consensus state: current_term: 1 leader_uuid: "7f5383ff5d894ed3b55c11b86af8af37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } } }
I20260808 11:37:40.957763 28619 tablet_replica.cc:442] T c48b211d295c49b694f5823af68a1b6f P 7f5383ff5d894ed3b55c11b86af8af37: This TxnStatusTablet replica's current role is: LEADER
I20260808 11:37:40.958447 28623 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 11:37:40.958734 28623 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 11:37:40.959200 27729 catalog_manager.cc:5495] T c48b211d295c49b694f5823af68a1b6f P 7f5383ff5d894ed3b55c11b86af8af37 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f5383ff5d894ed3b55c11b86af8af37 (127.24.76.67). New cstate: current_term: 1 leader_uuid: "7f5383ff5d894ed3b55c11b86af8af37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } health_report { overall_health: HEALTHY } } }
I20260808 11:37:40.985214 28572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:40.979817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47938 (local address 127.24.76.67:39691)
0808 11:37:40.979971 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:40.979977 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:40.979998 (+ 21us) server_negotiation.cc:407] Connection header received
0808 11:37:40.980336 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:40.980342 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:40.980434 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:40.980553 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:37:40.980906 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.981725 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.983083 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.983539 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.983648 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:40.984084 (+ 436us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:37:40.984594 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:37:40.984730 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:40.984822 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 11:37:40.984910 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 11:37:40.985235 28609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:40.979675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:39691 (local address 127.0.0.1:47938)
0808 11:37:40.979925 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:40.979944 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:40.980084 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:40.980599 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:40.980603 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:40.980615 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 11:37:40.980783 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:40.980792 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.981850 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.981856 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:40.982941 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:40.982961 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:40.983686 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:40.983692 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:40.983838 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:40.983901 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 11:37:40.984705 (+ 804us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 11:37:40.984726 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 11:37:40.984804 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 11:37:40.984969 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260808 11:37:40.992424 28219 raft_consensus.cc:1261] T c48b211d295c49b694f5823af68a1b6f P 30ff76bb491e45d88aafce3ec1eb4699 [term 1 FOLLOWER]: Refusing update from remote peer 7f5383ff5d894ed3b55c11b86af8af37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:37:40.992461 27970 raft_consensus.cc:1261] T c48b211d295c49b694f5823af68a1b6f P 3e8cee1d98e04a58a38583a87cd4acf5 [term 1 FOLLOWER]: Refusing update from remote peer 7f5383ff5d894ed3b55c11b86af8af37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:37:40.993359 28619 consensus_queue.cc:1035] T c48b211d295c49b694f5823af68a1b6f P 7f5383ff5d894ed3b55c11b86af8af37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:37:40.993706 28590 consensus_queue.cc:1035] T c48b211d295c49b694f5823af68a1b6f P 7f5383ff5d894ed3b55c11b86af8af37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:37:41.002864 28588 tablet_replica.cc:439] T c48b211d295c49b694f5823af68a1b6f P 30ff76bb491e45d88aafce3ec1eb4699: TxnStatusTablet state changed. Reason: New leader 7f5383ff5d894ed3b55c11b86af8af37. Latest consensus state: current_term: 1 leader_uuid: "7f5383ff5d894ed3b55c11b86af8af37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } } }
I20260808 11:37:41.003340 28588 tablet_replica.cc:442] T c48b211d295c49b694f5823af68a1b6f P 30ff76bb491e45d88aafce3ec1eb4699: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:37:41.003773 28589 tablet_replica.cc:439] T c48b211d295c49b694f5823af68a1b6f P 3e8cee1d98e04a58a38583a87cd4acf5: TxnStatusTablet state changed. Reason: New leader 7f5383ff5d894ed3b55c11b86af8af37. Latest consensus state: current_term: 1 leader_uuid: "7f5383ff5d894ed3b55c11b86af8af37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } } }
I20260808 11:37:41.004078 28589 tablet_replica.cc:442] T c48b211d295c49b694f5823af68a1b6f P 3e8cee1d98e04a58a38583a87cd4acf5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:37:41.009729 28588 tablet_replica.cc:439] T c48b211d295c49b694f5823af68a1b6f P 30ff76bb491e45d88aafce3ec1eb4699: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7f5383ff5d894ed3b55c11b86af8af37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } } }
I20260808 11:37:41.010027 28588 tablet_replica.cc:442] T c48b211d295c49b694f5823af68a1b6f P 30ff76bb491e45d88aafce3ec1eb4699: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:37:41.013110 28589 tablet_replica.cc:439] T c48b211d295c49b694f5823af68a1b6f P 3e8cee1d98e04a58a38583a87cd4acf5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7f5383ff5d894ed3b55c11b86af8af37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } } }
I20260808 11:37:41.013350 28589 tablet_replica.cc:442] T c48b211d295c49b694f5823af68a1b6f P 3e8cee1d98e04a58a38583a87cd4acf5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:37:41.014735 28590 tablet_replica.cc:439] T c48b211d295c49b694f5823af68a1b6f P 7f5383ff5d894ed3b55c11b86af8af37: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7f5383ff5d894ed3b55c11b86af8af37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } } }
I20260808 11:37:41.015057 28590 tablet_replica.cc:442] T c48b211d295c49b694f5823af68a1b6f P 7f5383ff5d894ed3b55c11b86af8af37: This TxnStatusTablet replica's current role is: LEADER
I20260808 11:37:41.015972 28590 tablet_replica.cc:439] T c48b211d295c49b694f5823af68a1b6f P 7f5383ff5d894ed3b55c11b86af8af37: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7f5383ff5d894ed3b55c11b86af8af37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } } }
I20260808 11:37:41.016196 28590 tablet_replica.cc:442] T c48b211d295c49b694f5823af68a1b6f P 7f5383ff5d894ed3b55c11b86af8af37: This TxnStatusTablet replica's current role is: LEADER
I20260808 11:37:41.028553 28618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:41.022513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60284 (local address 127.24.76.66:37571)
0808 11:37:41.022700 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:41.022706 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:41.023007 (+ 301us) server_negotiation.cc:407] Connection header received
0808 11:37:41.023219 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:41.023225 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:41.023322 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:41.023466 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:37:41.023844 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:41.024949 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:41.026295 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:41.026944 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:41.027067 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:41.027375 (+ 308us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:37:41.027998 (+ 623us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:37:41.028138 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:41.028184 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 11:37:41.028263 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 11:37:41.034386 28588 consensus_queue.cc:1035] T 042f264de9f0422b89fe3176919724d2 P 30ff76bb491e45d88aafce3ec1eb4699 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:37:41.038143 28588 consensus_queue.cc:1035] T 042f264de9f0422b89fe3176919724d2 P 30ff76bb491e45d88aafce3ec1eb4699 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:37:41.053632 28572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:41.040102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:34603 (local address 127.24.76.67:39691)
0808 11:37:41.040320 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:41.040327 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:41.046755 (+ 6428us) server_negotiation.cc:407] Connection header received
0808 11:37:41.046976 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:41.046981 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:41.047074 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:41.047162 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:37:41.047976 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:41.049124 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:41.051193 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:41.051925 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:41.052025 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:41.052847 (+ 822us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:37:41.052974 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:37:41.053043 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:41.053264 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 11:37:41.053344 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 11:37:41.054375 28651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:41.039597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:39691 (local address 127.24.76.66:34603)
0808 11:37:41.046637 (+ 7040us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:41.046659 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:41.046818 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:41.047334 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:41.047340 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:41.047353 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 11:37:41.047793 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:41.047806 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:41.049312 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:41.049317 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:41.051034 (+ 1717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:41.051049 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:41.052345 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:41.052352 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:41.052536 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:41.052607 (+ 71us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 11:37:41.053125 (+ 518us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 11:37:41.053135 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 11:37:41.053847 (+ 712us) client_negotiation.cc:240] Negotiation successful
0808 11:37:41.054012 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6857,"thread_start_us":6710,"threads_started":1}
I20260808 11:37:41.098188 28618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:41.087424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:38331 (local address 127.24.76.66:37571)
0808 11:37:41.087882 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:41.087889 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:41.092247 (+ 4358us) server_negotiation.cc:407] Connection header received
0808 11:37:41.092508 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:41.092514 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:41.092585 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:41.092657 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:37:41.093751 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:41.094628 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:41.096449 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:41.096884 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:41.096995 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:41.097460 (+ 465us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:37:41.097565 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:37:41.097635 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:41.097790 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 11:37:41.097874 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328}
I20260808 11:37:41.098379 28658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:41.088038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:37571 (local address 127.24.76.67:38331)
0808 11:37:41.092149 (+ 4111us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:41.092171 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:41.092321 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:41.092693 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:41.092697 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:41.092710 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 11:37:41.093599 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:41.093610 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:41.094758 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:41.094764 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:41.096305 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:41.096318 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:41.097082 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:41.097089 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:41.097217 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:41.097280 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 11:37:41.097667 (+ 387us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 11:37:41.097687 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 11:37:41.097763 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 11:37:41.097943 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3905,"thread_start_us":4271,"threads_started":1}
I20260808 11:37:41.145943 27723 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:37:41.146248 27723 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:37:41.150103 27723 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1d2e375a1ba543f0ae765fab7a56ca28: Sending DeleteTablet for 3 replicas of tablet 042f264de9f0422b89fe3176919724d2
I20260808 11:37:41.151310 27944 tablet_service.cc:1514] Processing DeleteTablet for tablet 042f264de9f0422b89fe3176919724d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:37:41 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:36300
I20260808 11:37:41.151364 28409 tablet_service.cc:1514] Processing DeleteTablet for tablet 042f264de9f0422b89fe3176919724d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:37:41 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:47926
I20260808 11:37:41.152637 28174 tablet_service.cc:1514] Processing DeleteTablet for tablet 042f264de9f0422b89fe3176919724d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:37:41 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:60272
I20260808 11:37:41.157246 27723 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60416:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:37:41.158242 27723 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:37:41.165341 28670 tablet_replica.cc:330] T 042f264de9f0422b89fe3176919724d2 P 3e8cee1d98e04a58a38583a87cd4acf5: stopping tablet replica
I20260808 11:37:41.166314 28407 tablet_service.cc:1467] Processing CreateTablet for tablet 24d5f2bd03734ff28037e17ff9632ea4 (DEFAULT_TABLE table=test-table [id=25252925d96d46e9b614353f0d7489b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:37:41.166462 28670 raft_consensus.cc:2229] T 042f264de9f0422b89fe3176919724d2 P 3e8cee1d98e04a58a38583a87cd4acf5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:37:41.166947 28407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24d5f2bd03734ff28037e17ff9632ea4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:37:41.167092 28670 raft_consensus.cc:2258] T 042f264de9f0422b89fe3176919724d2 P 3e8cee1d98e04a58a38583a87cd4acf5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:37:41.169148 28670 ts_tablet_manager.cc:1905] T 042f264de9f0422b89fe3176919724d2 P 3e8cee1d98e04a58a38583a87cd4acf5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:37:41.170670 28173 tablet_service.cc:1467] Processing CreateTablet for tablet 24d5f2bd03734ff28037e17ff9632ea4 (DEFAULT_TABLE table=test-table [id=25252925d96d46e9b614353f0d7489b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:37:41.171238 28173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24d5f2bd03734ff28037e17ff9632ea4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:37:41.175653 27944 tablet_service.cc:1467] Processing CreateTablet for tablet 24d5f2bd03734ff28037e17ff9632ea4 (DEFAULT_TABLE table=test-table [id=25252925d96d46e9b614353f0d7489b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:37:41.176064 27944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24d5f2bd03734ff28037e17ff9632ea4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:37:41.179718 28580 tablet_bootstrap.cc:492] T 24d5f2bd03734ff28037e17ff9632ea4 P 7f5383ff5d894ed3b55c11b86af8af37: Bootstrap starting.
I20260808 11:37:41.181427 28580 tablet_bootstrap.cc:654] T 24d5f2bd03734ff28037e17ff9632ea4 P 7f5383ff5d894ed3b55c11b86af8af37: Neither blocks nor log segments found. Creating new log.
I20260808 11:37:41.187446 28576 tablet_bootstrap.cc:492] T 24d5f2bd03734ff28037e17ff9632ea4 P 30ff76bb491e45d88aafce3ec1eb4699: Bootstrap starting.
I20260808 11:37:41.189428 28576 tablet_bootstrap.cc:654] T 24d5f2bd03734ff28037e17ff9632ea4 P 30ff76bb491e45d88aafce3ec1eb4699: Neither blocks nor log segments found. Creating new log.
I20260808 11:37:41.197186 28577 tablet_bootstrap.cc:492] T 24d5f2bd03734ff28037e17ff9632ea4 P 3e8cee1d98e04a58a38583a87cd4acf5: Bootstrap starting.
I20260808 11:37:41.198931 28577 tablet_bootstrap.cc:654] T 24d5f2bd03734ff28037e17ff9632ea4 P 3e8cee1d98e04a58a38583a87cd4acf5: Neither blocks nor log segments found. Creating new log.
I20260808 11:37:41.209731 28576 tablet_bootstrap.cc:492] T 24d5f2bd03734ff28037e17ff9632ea4 P 30ff76bb491e45d88aafce3ec1eb4699: No bootstrap required, opened a new log
I20260808 11:37:41.210121 28576 ts_tablet_manager.cc:1397] T 24d5f2bd03734ff28037e17ff9632ea4 P 30ff76bb491e45d88aafce3ec1eb4699: Time spent bootstrapping tablet: real 0.023s user 0.007s sys 0.002s
I20260808 11:37:41.211031 28576 raft_consensus.cc:348] T 24d5f2bd03734ff28037e17ff9632ea4 P 30ff76bb491e45d88aafce3ec1eb4699 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } }
I20260808 11:37:41.211426 28576 raft_consensus.cc:374] T 24d5f2bd03734ff28037e17ff9632ea4 P 30ff76bb491e45d88aafce3ec1eb4699 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:37:41.211552 28576 raft_consensus.cc:729] T 24d5f2bd03734ff28037e17ff9632ea4 P 30ff76bb491e45d88aafce3ec1eb4699 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30ff76bb491e45d88aafce3ec1eb4699, State: Initialized, Role: FOLLOWER
I20260808 11:37:41.211570 28580 tablet_bootstrap.cc:492] T 24d5f2bd03734ff28037e17ff9632ea4 P 7f5383ff5d894ed3b55c11b86af8af37: No bootstrap required, opened a new log
I20260808 11:37:41.211736 28580 ts_tablet_manager.cc:1397] T 24d5f2bd03734ff28037e17ff9632ea4 P 7f5383ff5d894ed3b55c11b86af8af37: Time spent bootstrapping tablet: real 0.032s user 0.003s sys 0.008s
I20260808 11:37:41.211956 28576 consensus_queue.cc:260] T 24d5f2bd03734ff28037e17ff9632ea4 P 30ff76bb491e45d88aafce3ec1eb4699 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } }
I20260808 11:37:41.212625 28580 raft_consensus.cc:348] T 24d5f2bd03734ff28037e17ff9632ea4 P 7f5383ff5d894ed3b55c11b86af8af37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } }
I20260808 11:37:41.212881 28580 raft_consensus.cc:374] T 24d5f2bd03734ff28037e17ff9632ea4 P 7f5383ff5d894ed3b55c11b86af8af37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:37:41.212970 28580 raft_consensus.cc:729] T 24d5f2bd03734ff28037e17ff9632ea4 P 7f5383ff5d894ed3b55c11b86af8af37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f5383ff5d894ed3b55c11b86af8af37, State: Initialized, Role: FOLLOWER
I20260808 11:37:41.213363 28580 consensus_queue.cc:260] T 24d5f2bd03734ff28037e17ff9632ea4 P 7f5383ff5d894ed3b55c11b86af8af37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } }
I20260808 11:37:41.216473 28576 ts_tablet_manager.cc:1428] T 24d5f2bd03734ff28037e17ff9632ea4 P 30ff76bb491e45d88aafce3ec1eb4699: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260808 11:37:41.216846 28580 ts_tablet_manager.cc:1428] T 24d5f2bd03734ff28037e17ff9632ea4 P 7f5383ff5d894ed3b55c11b86af8af37: Time spent starting tablet: real 0.005s user 0.002s sys 0.001s
I20260808 11:37:41.223027 28671 tablet_replica.cc:330] T 042f264de9f0422b89fe3176919724d2 P 30ff76bb491e45d88aafce3ec1eb4699: stopping tablet replica
I20260808 11:37:41.223446 28671 raft_consensus.cc:2229] T 042f264de9f0422b89fe3176919724d2 P 30ff76bb491e45d88aafce3ec1eb4699 [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:37:41.224108 28671 raft_consensus.cc:2258] T 042f264de9f0422b89fe3176919724d2 P 30ff76bb491e45d88aafce3ec1eb4699 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:37:41.224910 28667 tablet_replica.cc:330] T 042f264de9f0422b89fe3176919724d2 P 7f5383ff5d894ed3b55c11b86af8af37: stopping tablet replica
I20260808 11:37:41.225257 28667 raft_consensus.cc:2229] T 042f264de9f0422b89fe3176919724d2 P 7f5383ff5d894ed3b55c11b86af8af37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:37:41.225549 28667 raft_consensus.cc:2258] T 042f264de9f0422b89fe3176919724d2 P 7f5383ff5d894ed3b55c11b86af8af37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:37:41.226239 28670 ts_tablet_manager.cc:1918] T 042f264de9f0422b89fe3176919724d2 P 3e8cee1d98e04a58a38583a87cd4acf5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 11:37:41.226492 28670 log.cc:1196] T 042f264de9f0422b89fe3176919724d2 P 3e8cee1d98e04a58a38583a87cd4acf5: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-0/wal/wals/042f264de9f0422b89fe3176919724d2
I20260808 11:37:41.227069 28670 ts_tablet_manager.cc:1939] T 042f264de9f0422b89fe3176919724d2 P 3e8cee1d98e04a58a38583a87cd4acf5: Deleting consensus metadata
I20260808 11:37:41.227840 28671 ts_tablet_manager.cc:1905] T 042f264de9f0422b89fe3176919724d2 P 30ff76bb491e45d88aafce3ec1eb4699: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:37:41.226548 28577 tablet_bootstrap.cc:492] T 24d5f2bd03734ff28037e17ff9632ea4 P 3e8cee1d98e04a58a38583a87cd4acf5: No bootstrap required, opened a new log
I20260808 11:37:41.229378 28577 ts_tablet_manager.cc:1397] T 24d5f2bd03734ff28037e17ff9632ea4 P 3e8cee1d98e04a58a38583a87cd4acf5: Time spent bootstrapping tablet: real 0.032s user 0.009s sys 0.003s
I20260808 11:37:41.230170 28577 raft_consensus.cc:348] T 24d5f2bd03734ff28037e17ff9632ea4 P 3e8cee1d98e04a58a38583a87cd4acf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } }
I20260808 11:37:41.230489 28577 raft_consensus.cc:374] T 24d5f2bd03734ff28037e17ff9632ea4 P 3e8cee1d98e04a58a38583a87cd4acf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:37:41.230608 28577 raft_consensus.cc:729] T 24d5f2bd03734ff28037e17ff9632ea4 P 3e8cee1d98e04a58a38583a87cd4acf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8cee1d98e04a58a38583a87cd4acf5, State: Initialized, Role: FOLLOWER
I20260808 11:37:41.230953 28577 consensus_queue.cc:260] T 24d5f2bd03734ff28037e17ff9632ea4 P 3e8cee1d98e04a58a38583a87cd4acf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } }
I20260808 11:37:41.232239 27705 catalog_manager.cc:4841] TS 3e8cee1d98e04a58a38583a87cd4acf5 (127.24.76.65:34007): tablet 042f264de9f0422b89fe3176919724d2 (table test-table [id=abe5d416d2d7428aa0ca3d0387a4db1c]) successfully deleted
I20260808 11:37:41.232928 28577 ts_tablet_manager.cc:1428] T 24d5f2bd03734ff28037e17ff9632ea4 P 3e8cee1d98e04a58a38583a87cd4acf5: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260808 11:37:41.235271 28667 ts_tablet_manager.cc:1905] T 042f264de9f0422b89fe3176919724d2 P 7f5383ff5d894ed3b55c11b86af8af37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:37:41.240486 28671 ts_tablet_manager.cc:1918] T 042f264de9f0422b89fe3176919724d2 P 30ff76bb491e45d88aafce3ec1eb4699: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 11:37:41.240718 28671 log.cc:1196] T 042f264de9f0422b89fe3176919724d2 P 30ff76bb491e45d88aafce3ec1eb4699: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-1/wal/wals/042f264de9f0422b89fe3176919724d2
I20260808 11:37:41.241284 28671 ts_tablet_manager.cc:1939] T 042f264de9f0422b89fe3176919724d2 P 30ff76bb491e45d88aafce3ec1eb4699: Deleting consensus metadata
I20260808 11:37:41.242718 28667 ts_tablet_manager.cc:1918] T 042f264de9f0422b89fe3176919724d2 P 7f5383ff5d894ed3b55c11b86af8af37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 11:37:41.242900 28667 log.cc:1196] T 042f264de9f0422b89fe3176919724d2 P 7f5383ff5d894ed3b55c11b86af8af37: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-2/wal/wals/042f264de9f0422b89fe3176919724d2
I20260808 11:37:41.243238 27699 catalog_manager.cc:4841] TS 30ff76bb491e45d88aafce3ec1eb4699 (127.24.76.66:37571): tablet 042f264de9f0422b89fe3176919724d2 (table test-table [id=abe5d416d2d7428aa0ca3d0387a4db1c]) successfully deleted
I20260808 11:37:41.243460 28667 ts_tablet_manager.cc:1939] T 042f264de9f0422b89fe3176919724d2 P 7f5383ff5d894ed3b55c11b86af8af37: Deleting consensus metadata
I20260808 11:37:41.245072 27699 catalog_manager.cc:4841] TS 7f5383ff5d894ed3b55c11b86af8af37 (127.24.76.67:39691): tablet 042f264de9f0422b89fe3176919724d2 (table test-table [id=abe5d416d2d7428aa0ca3d0387a4db1c]) successfully deleted
I20260808 11:37:41.427924 28661 raft_consensus.cc:482] T 24d5f2bd03734ff28037e17ff9632ea4 P 30ff76bb491e45d88aafce3ec1eb4699 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:37:41.428206 28661 raft_consensus.cc:504] T 24d5f2bd03734ff28037e17ff9632ea4 P 30ff76bb491e45d88aafce3ec1eb4699 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } }
I20260808 11:37:41.429217 28661 leader_election.cc:288] T 24d5f2bd03734ff28037e17ff9632ea4 P 30ff76bb491e45d88aafce3ec1eb4699 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e8cee1d98e04a58a38583a87cd4acf5 (127.24.76.65:34007), 7f5383ff5d894ed3b55c11b86af8af37 (127.24.76.67:39691)
I20260808 11:37:41.429678 28452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24d5f2bd03734ff28037e17ff9632ea4" candidate_uuid: "30ff76bb491e45d88aafce3ec1eb4699" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f5383ff5d894ed3b55c11b86af8af37" is_pre_election: true
I20260808 11:37:41.429785 27970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24d5f2bd03734ff28037e17ff9632ea4" candidate_uuid: "30ff76bb491e45d88aafce3ec1eb4699" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" is_pre_election: true
I20260808 11:37:41.429940 28452 raft_consensus.cc:2454] T 24d5f2bd03734ff28037e17ff9632ea4 P 7f5383ff5d894ed3b55c11b86af8af37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30ff76bb491e45d88aafce3ec1eb4699 in term 0.
I20260808 11:37:41.430176 27970 raft_consensus.cc:2454] T 24d5f2bd03734ff28037e17ff9632ea4 P 3e8cee1d98e04a58a38583a87cd4acf5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30ff76bb491e45d88aafce3ec1eb4699 in term 0.
I20260808 11:37:41.430363 28089 leader_election.cc:302] T 24d5f2bd03734ff28037e17ff9632ea4 P 30ff76bb491e45d88aafce3ec1eb4699 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30ff76bb491e45d88aafce3ec1eb4699, 7f5383ff5d894ed3b55c11b86af8af37; no voters:
I20260808 11:37:41.430718 28661 raft_consensus.cc:2781] T 24d5f2bd03734ff28037e17ff9632ea4 P 30ff76bb491e45d88aafce3ec1eb4699 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:37:41.430861 28661 raft_consensus.cc:482] T 24d5f2bd03734ff28037e17ff9632ea4 P 30ff76bb491e45d88aafce3ec1eb4699 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:37:41.430963 28661 raft_consensus.cc:3037] T 24d5f2bd03734ff28037e17ff9632ea4 P 30ff76bb491e45d88aafce3ec1eb4699 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:37:41.433643 28661 raft_consensus.cc:504] T 24d5f2bd03734ff28037e17ff9632ea4 P 30ff76bb491e45d88aafce3ec1eb4699 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } }
I20260808 11:37:41.434213 28661 leader_election.cc:288] T 24d5f2bd03734ff28037e17ff9632ea4 P 30ff76bb491e45d88aafce3ec1eb4699 [CANDIDATE]: Term 1 election: Requested vote from peers 3e8cee1d98e04a58a38583a87cd4acf5 (127.24.76.65:34007), 7f5383ff5d894ed3b55c11b86af8af37 (127.24.76.67:39691)
I20260808 11:37:41.434736 27970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24d5f2bd03734ff28037e17ff9632ea4" candidate_uuid: "30ff76bb491e45d88aafce3ec1eb4699" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8cee1d98e04a58a38583a87cd4acf5"
I20260808 11:37:41.434926 27970 raft_consensus.cc:3037] T 24d5f2bd03734ff28037e17ff9632ea4 P 3e8cee1d98e04a58a38583a87cd4acf5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:37:41.434898 28452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24d5f2bd03734ff28037e17ff9632ea4" candidate_uuid: "30ff76bb491e45d88aafce3ec1eb4699" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f5383ff5d894ed3b55c11b86af8af37"
I20260808 11:37:41.435144 28452 raft_consensus.cc:3037] T 24d5f2bd03734ff28037e17ff9632ea4 P 7f5383ff5d894ed3b55c11b86af8af37 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:37:41.437237 27970 raft_consensus.cc:2454] T 24d5f2bd03734ff28037e17ff9632ea4 P 3e8cee1d98e04a58a38583a87cd4acf5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30ff76bb491e45d88aafce3ec1eb4699 in term 1.
I20260808 11:37:41.437704 28094 leader_election.cc:302] T 24d5f2bd03734ff28037e17ff9632ea4 P 30ff76bb491e45d88aafce3ec1eb4699 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30ff76bb491e45d88aafce3ec1eb4699, 3e8cee1d98e04a58a38583a87cd4acf5; no voters:
I20260808 11:37:41.438033 28661 raft_consensus.cc:2781] T 24d5f2bd03734ff28037e17ff9632ea4 P 30ff76bb491e45d88aafce3ec1eb4699 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:37:41.438198 28661 raft_consensus.cc:686] T 24d5f2bd03734ff28037e17ff9632ea4 P 30ff76bb491e45d88aafce3ec1eb4699 [term 1 LEADER]: Becoming Leader. State: Replica: 30ff76bb491e45d88aafce3ec1eb4699, State: Running, Role: LEADER
I20260808 11:37:41.438549 28452 raft_consensus.cc:2454] T 24d5f2bd03734ff28037e17ff9632ea4 P 7f5383ff5d894ed3b55c11b86af8af37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30ff76bb491e45d88aafce3ec1eb4699 in term 1.
I20260808 11:37:41.438512 28661 consensus_queue.cc:237] T 24d5f2bd03734ff28037e17ff9632ea4 P 30ff76bb491e45d88aafce3ec1eb4699 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } }
I20260808 11:37:41.441761 27729 catalog_manager.cc:5495] T 24d5f2bd03734ff28037e17ff9632ea4 P 30ff76bb491e45d88aafce3ec1eb4699 reported cstate change: term changed from 0 to 1, leader changed from <none> to 30ff76bb491e45d88aafce3ec1eb4699 (127.24.76.66). New cstate: current_term: 1 leader_uuid: "30ff76bb491e45d88aafce3ec1eb4699" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } health_report { overall_health: UNKNOWN } } }
I20260808 11:37:41.504034 27970 raft_consensus.cc:1261] T 24d5f2bd03734ff28037e17ff9632ea4 P 3e8cee1d98e04a58a38583a87cd4acf5 [term 1 FOLLOWER]: Refusing update from remote peer 30ff76bb491e45d88aafce3ec1eb4699: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:37:41.504020 28452 raft_consensus.cc:1261] T 24d5f2bd03734ff28037e17ff9632ea4 P 7f5383ff5d894ed3b55c11b86af8af37 [term 1 FOLLOWER]: Refusing update from remote peer 30ff76bb491e45d88aafce3ec1eb4699: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:37:41.504596 28595 consensus_queue.cc:1035] T 24d5f2bd03734ff28037e17ff9632ea4 P 30ff76bb491e45d88aafce3ec1eb4699 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:37:41.505151 28661 consensus_queue.cc:1035] T 24d5f2bd03734ff28037e17ff9632ea4 P 30ff76bb491e45d88aafce3ec1eb4699 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 11:37:41.531342 28629 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 11:37:43.114889 27723 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:37:43.115255 27723 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:37:43.120316 27723 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1d2e375a1ba543f0ae765fab7a56ca28: Sending DeleteTablet for 3 replicas of tablet 24d5f2bd03734ff28037e17ff9632ea4
I20260808 11:37:43.121081 27944 tablet_service.cc:1514] Processing DeleteTablet for tablet 24d5f2bd03734ff28037e17ff9632ea4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:37:43 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:36300
I20260808 11:37:43.121730 28174 tablet_service.cc:1514] Processing DeleteTablet for tablet 24d5f2bd03734ff28037e17ff9632ea4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:37:43 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:60272
I20260808 11:37:43.121742 28409 tablet_service.cc:1514] Processing DeleteTablet for tablet 24d5f2bd03734ff28037e17ff9632ea4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:37:43 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:47926
I20260808 11:37:43.122640 28728 tablet_replica.cc:330] T 24d5f2bd03734ff28037e17ff9632ea4 P 3e8cee1d98e04a58a38583a87cd4acf5: stopping tablet replica
I20260808 11:37:43.122802 28728 raft_consensus.cc:2229] T 24d5f2bd03734ff28037e17ff9632ea4 P 3e8cee1d98e04a58a38583a87cd4acf5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:37:43.123045 28728 raft_consensus.cc:2258] T 24d5f2bd03734ff28037e17ff9632ea4 P 3e8cee1d98e04a58a38583a87cd4acf5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:37:43.123960 28728 ts_tablet_manager.cc:1905] T 24d5f2bd03734ff28037e17ff9632ea4 P 3e8cee1d98e04a58a38583a87cd4acf5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:37:43.131124 28728 ts_tablet_manager.cc:1918] T 24d5f2bd03734ff28037e17ff9632ea4 P 3e8cee1d98e04a58a38583a87cd4acf5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 11:37:43.131373 28728 log.cc:1196] T 24d5f2bd03734ff28037e17ff9632ea4 P 3e8cee1d98e04a58a38583a87cd4acf5: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-0/wal/wals/24d5f2bd03734ff28037e17ff9632ea4
I20260808 11:37:43.132083 28728 ts_tablet_manager.cc:1939] T 24d5f2bd03734ff28037e17ff9632ea4 P 3e8cee1d98e04a58a38583a87cd4acf5: Deleting consensus metadata
I20260808 11:37:43.133406 27705 catalog_manager.cc:4841] TS 3e8cee1d98e04a58a38583a87cd4acf5 (127.24.76.65:34007): tablet 24d5f2bd03734ff28037e17ff9632ea4 (table test-table [id=25252925d96d46e9b614353f0d7489b4]) successfully deleted
I20260808 11:37:43.134441 28729 tablet_replica.cc:330] T 24d5f2bd03734ff28037e17ff9632ea4 P 30ff76bb491e45d88aafce3ec1eb4699: stopping tablet replica
I20260808 11:37:43.134670 28729 raft_consensus.cc:2229] T 24d5f2bd03734ff28037e17ff9632ea4 P 30ff76bb491e45d88aafce3ec1eb4699 [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:37:43.135039 28729 raft_consensus.cc:2258] T 24d5f2bd03734ff28037e17ff9632ea4 P 30ff76bb491e45d88aafce3ec1eb4699 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:37:43.135782 28730 tablet_replica.cc:330] T 24d5f2bd03734ff28037e17ff9632ea4 P 7f5383ff5d894ed3b55c11b86af8af37: stopping tablet replica
I20260808 11:37:43.135957 28730 raft_consensus.cc:2229] T 24d5f2bd03734ff28037e17ff9632ea4 P 7f5383ff5d894ed3b55c11b86af8af37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:37:43.136209 28730 raft_consensus.cc:2258] T 24d5f2bd03734ff28037e17ff9632ea4 P 7f5383ff5d894ed3b55c11b86af8af37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:37:43.136994 28729 ts_tablet_manager.cc:1905] T 24d5f2bd03734ff28037e17ff9632ea4 P 30ff76bb491e45d88aafce3ec1eb4699: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:37:43.137142 28730 ts_tablet_manager.cc:1905] T 24d5f2bd03734ff28037e17ff9632ea4 P 7f5383ff5d894ed3b55c11b86af8af37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:37:43.141958 28729 ts_tablet_manager.cc:1918] T 24d5f2bd03734ff28037e17ff9632ea4 P 30ff76bb491e45d88aafce3ec1eb4699: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 11:37:43.142086 28729 log.cc:1196] T 24d5f2bd03734ff28037e17ff9632ea4 P 30ff76bb491e45d88aafce3ec1eb4699: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-1/wal/wals/24d5f2bd03734ff28037e17ff9632ea4
I20260808 11:37:43.142663 28729 ts_tablet_manager.cc:1939] T 24d5f2bd03734ff28037e17ff9632ea4 P 30ff76bb491e45d88aafce3ec1eb4699: Deleting consensus metadata
I20260808 11:37:43.143633 27699 catalog_manager.cc:4841] TS 30ff76bb491e45d88aafce3ec1eb4699 (127.24.76.66:37571): tablet 24d5f2bd03734ff28037e17ff9632ea4 (table test-table [id=25252925d96d46e9b614353f0d7489b4]) successfully deleted
I20260808 11:37:43.143754 28730 ts_tablet_manager.cc:1918] T 24d5f2bd03734ff28037e17ff9632ea4 P 7f5383ff5d894ed3b55c11b86af8af37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 11:37:43.143877 28730 log.cc:1196] T 24d5f2bd03734ff28037e17ff9632ea4 P 7f5383ff5d894ed3b55c11b86af8af37: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-2/wal/wals/24d5f2bd03734ff28037e17ff9632ea4
I20260808 11:37:43.144236 28730 ts_tablet_manager.cc:1939] T 24d5f2bd03734ff28037e17ff9632ea4 P 7f5383ff5d894ed3b55c11b86af8af37: Deleting consensus metadata
I20260808 11:37:43.145074 27699 catalog_manager.cc:4841] TS 7f5383ff5d894ed3b55c11b86af8af37 (127.24.76.67:39691): tablet 24d5f2bd03734ff28037e17ff9632ea4 (table test-table [id=25252925d96d46e9b614353f0d7489b4]) successfully deleted
Aug 08 11:37:43 dist-test-slave-tc2s krb5kdc[27653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189063, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 11:37:43 dist-test-slave-tc2s krb5kdc[27653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189063, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:37:43.182974 28746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:43.164904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36862 (local address 127.24.76.126:45521)
0808 11:37:43.165530 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:43.165535 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:43.165560 (+ 25us) server_negotiation.cc:407] Connection header received
0808 11:37:43.165702 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:43.165707 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:43.165772 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:43.165864 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:43.167276 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:43.168064 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:43.169290 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:43.169692 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:43.169801 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:43.177713 (+ 7912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:43.177719 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:43.177722 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:43.177756 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:43.180480 (+ 2724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:43.181143 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:43.181147 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:43.181149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:43.181279 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:43.181612 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:43.181614 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:43.181617 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:43.181883 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:43.182107 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:43.182478 (+ 371us) server_negotiation.cc:299] Negotiation successful
0808 11:37:43.182644 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":294,"threads_started":1}
I20260808 11:37:43.186471 27723 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36862:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:37:43.187139 27723 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:37:43.193153 27944 tablet_service.cc:1467] Processing CreateTablet for tablet 893aa81595ce4270ac2ce2636a5073cd (DEFAULT_TABLE table=test-table [id=f16589be3bce438f8b9d89246a9c66f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:37:43.193279 28174 tablet_service.cc:1467] Processing CreateTablet for tablet 893aa81595ce4270ac2ce2636a5073cd (DEFAULT_TABLE table=test-table [id=f16589be3bce438f8b9d89246a9c66f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:37:43.193481 28409 tablet_service.cc:1467] Processing CreateTablet for tablet 893aa81595ce4270ac2ce2636a5073cd (DEFAULT_TABLE table=test-table [id=f16589be3bce438f8b9d89246a9c66f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:37:43.193639 27944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 893aa81595ce4270ac2ce2636a5073cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:37:43.193653 28174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 893aa81595ce4270ac2ce2636a5073cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:37:43.193874 28409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 893aa81595ce4270ac2ce2636a5073cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:37:43.200538 28747 tablet_bootstrap.cc:492] T 893aa81595ce4270ac2ce2636a5073cd P 7f5383ff5d894ed3b55c11b86af8af37: Bootstrap starting.
I20260808 11:37:43.202518 28747 tablet_bootstrap.cc:654] T 893aa81595ce4270ac2ce2636a5073cd P 7f5383ff5d894ed3b55c11b86af8af37: Neither blocks nor log segments found. Creating new log.
I20260808 11:37:43.205015 28747 tablet_bootstrap.cc:492] T 893aa81595ce4270ac2ce2636a5073cd P 7f5383ff5d894ed3b55c11b86af8af37: No bootstrap required, opened a new log
I20260808 11:37:43.205160 28747 ts_tablet_manager.cc:1397] T 893aa81595ce4270ac2ce2636a5073cd P 7f5383ff5d894ed3b55c11b86af8af37: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 11:37:43.205945 28747 raft_consensus.cc:348] T 893aa81595ce4270ac2ce2636a5073cd P 7f5383ff5d894ed3b55c11b86af8af37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } }
I20260808 11:37:43.206305 28747 raft_consensus.cc:374] T 893aa81595ce4270ac2ce2636a5073cd P 7f5383ff5d894ed3b55c11b86af8af37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:37:43.206528 28747 raft_consensus.cc:729] T 893aa81595ce4270ac2ce2636a5073cd P 7f5383ff5d894ed3b55c11b86af8af37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f5383ff5d894ed3b55c11b86af8af37, State: Initialized, Role: FOLLOWER
I20260808 11:37:43.206805 28747 consensus_queue.cc:260] T 893aa81595ce4270ac2ce2636a5073cd P 7f5383ff5d894ed3b55c11b86af8af37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } }
I20260808 11:37:43.207458 28747 ts_tablet_manager.cc:1428] T 893aa81595ce4270ac2ce2636a5073cd P 7f5383ff5d894ed3b55c11b86af8af37: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 11:37:43.208418 28750 tablet_bootstrap.cc:492] T 893aa81595ce4270ac2ce2636a5073cd P 30ff76bb491e45d88aafce3ec1eb4699: Bootstrap starting.
I20260808 11:37:43.208541 28748 tablet_bootstrap.cc:492] T 893aa81595ce4270ac2ce2636a5073cd P 3e8cee1d98e04a58a38583a87cd4acf5: Bootstrap starting.
I20260808 11:37:43.210805 28748 tablet_bootstrap.cc:654] T 893aa81595ce4270ac2ce2636a5073cd P 3e8cee1d98e04a58a38583a87cd4acf5: Neither blocks nor log segments found. Creating new log.
I20260808 11:37:43.211051 28750 tablet_bootstrap.cc:654] T 893aa81595ce4270ac2ce2636a5073cd P 30ff76bb491e45d88aafce3ec1eb4699: Neither blocks nor log segments found. Creating new log.
I20260808 11:37:43.213415 28748 tablet_bootstrap.cc:492] T 893aa81595ce4270ac2ce2636a5073cd P 3e8cee1d98e04a58a38583a87cd4acf5: No bootstrap required, opened a new log
I20260808 11:37:43.213613 28748 ts_tablet_manager.cc:1397] T 893aa81595ce4270ac2ce2636a5073cd P 3e8cee1d98e04a58a38583a87cd4acf5: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 11:37:43.214412 28748 raft_consensus.cc:348] T 893aa81595ce4270ac2ce2636a5073cd P 3e8cee1d98e04a58a38583a87cd4acf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } }
I20260808 11:37:43.214686 28748 raft_consensus.cc:374] T 893aa81595ce4270ac2ce2636a5073cd P 3e8cee1d98e04a58a38583a87cd4acf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:37:43.214910 28748 raft_consensus.cc:729] T 893aa81595ce4270ac2ce2636a5073cd P 3e8cee1d98e04a58a38583a87cd4acf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8cee1d98e04a58a38583a87cd4acf5, State: Initialized, Role: FOLLOWER
I20260808 11:37:43.215049 28750 tablet_bootstrap.cc:492] T 893aa81595ce4270ac2ce2636a5073cd P 30ff76bb491e45d88aafce3ec1eb4699: No bootstrap required, opened a new log
I20260808 11:37:43.215231 28750 ts_tablet_manager.cc:1397] T 893aa81595ce4270ac2ce2636a5073cd P 30ff76bb491e45d88aafce3ec1eb4699: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260808 11:37:43.215276 28748 consensus_queue.cc:260] T 893aa81595ce4270ac2ce2636a5073cd P 3e8cee1d98e04a58a38583a87cd4acf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } }
I20260808 11:37:43.216234 28748 ts_tablet_manager.cc:1428] T 893aa81595ce4270ac2ce2636a5073cd P 3e8cee1d98e04a58a38583a87cd4acf5: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 11:37:43.216151 28750 raft_consensus.cc:348] T 893aa81595ce4270ac2ce2636a5073cd P 30ff76bb491e45d88aafce3ec1eb4699 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } }
I20260808 11:37:43.216445 28750 raft_consensus.cc:374] T 893aa81595ce4270ac2ce2636a5073cd P 30ff76bb491e45d88aafce3ec1eb4699 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:37:43.216619 28750 raft_consensus.cc:729] T 893aa81595ce4270ac2ce2636a5073cd P 30ff76bb491e45d88aafce3ec1eb4699 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30ff76bb491e45d88aafce3ec1eb4699, State: Initialized, Role: FOLLOWER
I20260808 11:37:43.216889 28750 consensus_queue.cc:260] T 893aa81595ce4270ac2ce2636a5073cd P 30ff76bb491e45d88aafce3ec1eb4699 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } }
I20260808 11:37:43.217429 28750 ts_tablet_manager.cc:1428] T 893aa81595ce4270ac2ce2636a5073cd P 30ff76bb491e45d88aafce3ec1eb4699: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 11:37:43.236013 28686 raft_consensus.cc:482] T 893aa81595ce4270ac2ce2636a5073cd P 30ff76bb491e45d88aafce3ec1eb4699 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:37:43.236162 28686 raft_consensus.cc:504] T 893aa81595ce4270ac2ce2636a5073cd P 30ff76bb491e45d88aafce3ec1eb4699 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } }
I20260808 11:37:43.236825 28686 leader_election.cc:288] T 893aa81595ce4270ac2ce2636a5073cd P 30ff76bb491e45d88aafce3ec1eb4699 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e8cee1d98e04a58a38583a87cd4acf5 (127.24.76.65:34007), 7f5383ff5d894ed3b55c11b86af8af37 (127.24.76.67:39691)
I20260808 11:37:43.237306 27970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "893aa81595ce4270ac2ce2636a5073cd" candidate_uuid: "30ff76bb491e45d88aafce3ec1eb4699" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" is_pre_election: true
I20260808 11:37:43.237603 27970 raft_consensus.cc:2454] T 893aa81595ce4270ac2ce2636a5073cd P 3e8cee1d98e04a58a38583a87cd4acf5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30ff76bb491e45d88aafce3ec1eb4699 in term 0.
I20260808 11:37:43.237584 28450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "893aa81595ce4270ac2ce2636a5073cd" candidate_uuid: "30ff76bb491e45d88aafce3ec1eb4699" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f5383ff5d894ed3b55c11b86af8af37" is_pre_election: true
I20260808 11:37:43.237850 28450 raft_consensus.cc:2454] T 893aa81595ce4270ac2ce2636a5073cd P 7f5383ff5d894ed3b55c11b86af8af37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30ff76bb491e45d88aafce3ec1eb4699 in term 0.
I20260808 11:37:43.238137 28094 leader_election.cc:302] T 893aa81595ce4270ac2ce2636a5073cd P 30ff76bb491e45d88aafce3ec1eb4699 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30ff76bb491e45d88aafce3ec1eb4699, 3e8cee1d98e04a58a38583a87cd4acf5; no voters:
I20260808 11:37:43.238420 28686 raft_consensus.cc:2781] T 893aa81595ce4270ac2ce2636a5073cd P 30ff76bb491e45d88aafce3ec1eb4699 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:37:43.238564 28686 raft_consensus.cc:482] T 893aa81595ce4270ac2ce2636a5073cd P 30ff76bb491e45d88aafce3ec1eb4699 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:37:43.238648 28686 raft_consensus.cc:3037] T 893aa81595ce4270ac2ce2636a5073cd P 30ff76bb491e45d88aafce3ec1eb4699 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:37:43.240831 28686 raft_consensus.cc:504] T 893aa81595ce4270ac2ce2636a5073cd P 30ff76bb491e45d88aafce3ec1eb4699 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } }
I20260808 11:37:43.241379 28686 leader_election.cc:288] T 893aa81595ce4270ac2ce2636a5073cd P 30ff76bb491e45d88aafce3ec1eb4699 [CANDIDATE]: Term 1 election: Requested vote from peers 3e8cee1d98e04a58a38583a87cd4acf5 (127.24.76.65:34007), 7f5383ff5d894ed3b55c11b86af8af37 (127.24.76.67:39691)
I20260808 11:37:43.241787 28450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "893aa81595ce4270ac2ce2636a5073cd" candidate_uuid: "30ff76bb491e45d88aafce3ec1eb4699" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f5383ff5d894ed3b55c11b86af8af37"
I20260808 11:37:43.241832 27970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "893aa81595ce4270ac2ce2636a5073cd" candidate_uuid: "30ff76bb491e45d88aafce3ec1eb4699" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8cee1d98e04a58a38583a87cd4acf5"
I20260808 11:37:43.242071 28450 raft_consensus.cc:3037] T 893aa81595ce4270ac2ce2636a5073cd P 7f5383ff5d894ed3b55c11b86af8af37 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:37:43.242105 27970 raft_consensus.cc:3037] T 893aa81595ce4270ac2ce2636a5073cd P 3e8cee1d98e04a58a38583a87cd4acf5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:37:43.244567 27970 raft_consensus.cc:2454] T 893aa81595ce4270ac2ce2636a5073cd P 3e8cee1d98e04a58a38583a87cd4acf5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30ff76bb491e45d88aafce3ec1eb4699 in term 1.
I20260808 11:37:43.244716 28450 raft_consensus.cc:2454] T 893aa81595ce4270ac2ce2636a5073cd P 7f5383ff5d894ed3b55c11b86af8af37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30ff76bb491e45d88aafce3ec1eb4699 in term 1.
I20260808 11:37:43.244971 28094 leader_election.cc:302] T 893aa81595ce4270ac2ce2636a5073cd P 30ff76bb491e45d88aafce3ec1eb4699 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30ff76bb491e45d88aafce3ec1eb4699, 3e8cee1d98e04a58a38583a87cd4acf5; no voters:
I20260808 11:37:43.245244 28686 raft_consensus.cc:2781] T 893aa81595ce4270ac2ce2636a5073cd P 30ff76bb491e45d88aafce3ec1eb4699 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:37:43.245378 28686 raft_consensus.cc:686] T 893aa81595ce4270ac2ce2636a5073cd P 30ff76bb491e45d88aafce3ec1eb4699 [term 1 LEADER]: Becoming Leader. State: Replica: 30ff76bb491e45d88aafce3ec1eb4699, State: Running, Role: LEADER
I20260808 11:37:43.245661 28686 consensus_queue.cc:237] T 893aa81595ce4270ac2ce2636a5073cd P 30ff76bb491e45d88aafce3ec1eb4699 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } }
I20260808 11:37:43.248409 27723 catalog_manager.cc:5495] T 893aa81595ce4270ac2ce2636a5073cd P 30ff76bb491e45d88aafce3ec1eb4699 reported cstate change: term changed from 0 to 1, leader changed from <none> to 30ff76bb491e45d88aafce3ec1eb4699 (127.24.76.66). New cstate: current_term: 1 leader_uuid: "30ff76bb491e45d88aafce3ec1eb4699" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } health_report { overall_health: UNKNOWN } } }
I20260808 11:37:43.275149 28758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:43.269273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48122 (local address 127.24.76.66:37571)
0808 11:37:43.270156 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:43.270161 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:43.270180 (+ 19us) server_negotiation.cc:407] Connection header received
0808 11:37:43.270228 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:43.270232 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:43.270289 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:43.270371 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:37:43.270833 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:43.271746 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:43.273180 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:43.273699 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:43.273809 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:43.274409 (+ 600us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:37:43.274525 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:37:43.274604 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:43.274747 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 11:37:43.274821 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":730,"thread_start_us":560,"threads_started":1}
I20260808 11:37:43.284917 28450 raft_consensus.cc:1261] T 893aa81595ce4270ac2ce2636a5073cd P 7f5383ff5d894ed3b55c11b86af8af37 [term 1 FOLLOWER]: Refusing update from remote peer 30ff76bb491e45d88aafce3ec1eb4699: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:37:43.285063 27970 raft_consensus.cc:1261] T 893aa81595ce4270ac2ce2636a5073cd P 3e8cee1d98e04a58a38583a87cd4acf5 [term 1 FOLLOWER]: Refusing update from remote peer 30ff76bb491e45d88aafce3ec1eb4699: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:37:43.285885 28719 consensus_queue.cc:1035] T 893aa81595ce4270ac2ce2636a5073cd P 30ff76bb491e45d88aafce3ec1eb4699 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:37:43.286216 28686 consensus_queue.cc:1035] T 893aa81595ce4270ac2ce2636a5073cd P 30ff76bb491e45d88aafce3ec1eb4699 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:37:43.292754 28697 mvcc.cc:205] Tried to move back new op lower bound from 7316230403209420800 to 7316230403056996352. Current Snapshot: MvccSnapshot[applied={T|T < 7316230403209420800}]
I20260808 11:37:43.341791 27729 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:37:43.342061 27729 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:37:43.344751 27729 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1d2e375a1ba543f0ae765fab7a56ca28: Sending DeleteTablet for 3 replicas of tablet 893aa81595ce4270ac2ce2636a5073cd
I20260808 11:37:43.345542 27944 tablet_service.cc:1514] Processing DeleteTablet for tablet 893aa81595ce4270ac2ce2636a5073cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:37:43 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:36300
I20260808 11:37:43.345677 28174 tablet_service.cc:1514] Processing DeleteTablet for tablet 893aa81595ce4270ac2ce2636a5073cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:37:43 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:60272
I20260808 11:37:43.345712 28409 tablet_service.cc:1514] Processing DeleteTablet for tablet 893aa81595ce4270ac2ce2636a5073cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:37:43 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:47926
I20260808 11:37:43.345927 28728 tablet_replica.cc:330] T 893aa81595ce4270ac2ce2636a5073cd P 3e8cee1d98e04a58a38583a87cd4acf5: stopping tablet replica
I20260808 11:37:43.345999 28729 tablet_replica.cc:330] T 893aa81595ce4270ac2ce2636a5073cd P 30ff76bb491e45d88aafce3ec1eb4699: stopping tablet replica
I20260808 11:37:43.346194 28728 raft_consensus.cc:2229] T 893aa81595ce4270ac2ce2636a5073cd P 3e8cee1d98e04a58a38583a87cd4acf5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:37:43.346216 28730 tablet_replica.cc:330] T 893aa81595ce4270ac2ce2636a5073cd P 7f5383ff5d894ed3b55c11b86af8af37: stopping tablet replica
I20260808 11:37:43.346243 28729 raft_consensus.cc:2229] T 893aa81595ce4270ac2ce2636a5073cd P 30ff76bb491e45d88aafce3ec1eb4699 [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:37:43.346478 28730 raft_consensus.cc:2229] T 893aa81595ce4270ac2ce2636a5073cd P 7f5383ff5d894ed3b55c11b86af8af37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:37:43.346439 28728 raft_consensus.cc:2258] T 893aa81595ce4270ac2ce2636a5073cd P 3e8cee1d98e04a58a38583a87cd4acf5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:37:43.346601 28729 raft_consensus.cc:2258] T 893aa81595ce4270ac2ce2636a5073cd P 30ff76bb491e45d88aafce3ec1eb4699 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:37:43.346695 28730 raft_consensus.cc:2258] T 893aa81595ce4270ac2ce2636a5073cd P 7f5383ff5d894ed3b55c11b86af8af37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:37:43.347707 28730 ts_tablet_manager.cc:1905] T 893aa81595ce4270ac2ce2636a5073cd P 7f5383ff5d894ed3b55c11b86af8af37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:37:43.347746 28728 ts_tablet_manager.cc:1905] T 893aa81595ce4270ac2ce2636a5073cd P 3e8cee1d98e04a58a38583a87cd4acf5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:37:43.347960 28729 ts_tablet_manager.cc:1905] T 893aa81595ce4270ac2ce2636a5073cd P 30ff76bb491e45d88aafce3ec1eb4699: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:37:43.350236 27729 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36862:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:37:43.351070 27729 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:37:43.353916 28730 ts_tablet_manager.cc:1918] T 893aa81595ce4270ac2ce2636a5073cd P 7f5383ff5d894ed3b55c11b86af8af37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 11:37:43.354206 28730 log.cc:1196] T 893aa81595ce4270ac2ce2636a5073cd P 7f5383ff5d894ed3b55c11b86af8af37: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-2/wal/wals/893aa81595ce4270ac2ce2636a5073cd
I20260808 11:37:43.354566 28730 ts_tablet_manager.cc:1939] T 893aa81595ce4270ac2ce2636a5073cd P 7f5383ff5d894ed3b55c11b86af8af37: Deleting consensus metadata
I20260808 11:37:43.355060 28728 ts_tablet_manager.cc:1918] T 893aa81595ce4270ac2ce2636a5073cd P 3e8cee1d98e04a58a38583a87cd4acf5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 11:37:43.355306 28728 log.cc:1196] T 893aa81595ce4270ac2ce2636a5073cd P 3e8cee1d98e04a58a38583a87cd4acf5: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-0/wal/wals/893aa81595ce4270ac2ce2636a5073cd
I20260808 11:37:43.355430 27699 catalog_manager.cc:4841] TS 7f5383ff5d894ed3b55c11b86af8af37 (127.24.76.67:39691): tablet 893aa81595ce4270ac2ce2636a5073cd (table test-table [id=f16589be3bce438f8b9d89246a9c66f5]) successfully deleted
I20260808 11:37:43.355922 28728 ts_tablet_manager.cc:1939] T 893aa81595ce4270ac2ce2636a5073cd P 3e8cee1d98e04a58a38583a87cd4acf5: Deleting consensus metadata
I20260808 11:37:43.357084 27705 catalog_manager.cc:4841] TS 3e8cee1d98e04a58a38583a87cd4acf5 (127.24.76.65:34007): tablet 893aa81595ce4270ac2ce2636a5073cd (table test-table [id=f16589be3bce438f8b9d89246a9c66f5]) successfully deleted
I20260808 11:37:43.357631 28729 ts_tablet_manager.cc:1918] T 893aa81595ce4270ac2ce2636a5073cd P 30ff76bb491e45d88aafce3ec1eb4699: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 11:37:43.357849 28729 log.cc:1196] T 893aa81595ce4270ac2ce2636a5073cd P 30ff76bb491e45d88aafce3ec1eb4699: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-1/wal/wals/893aa81595ce4270ac2ce2636a5073cd
I20260808 11:37:43.358388 28729 ts_tablet_manager.cc:1939] T 893aa81595ce4270ac2ce2636a5073cd P 30ff76bb491e45d88aafce3ec1eb4699: Deleting consensus metadata
I20260808 11:37:43.358305 27944 tablet_service.cc:1467] Processing CreateTablet for tablet 3dbd3ad800ea432c80e6827b892ebd13 (DEFAULT_TABLE table=test-table [id=ac3d135744ef4a688c5fb64c47111046]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:37:43.358748 27944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3dbd3ad800ea432c80e6827b892ebd13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:37:43.358686 28174 tablet_service.cc:1467] Processing CreateTablet for tablet 3dbd3ad800ea432c80e6827b892ebd13 (DEFAULT_TABLE table=test-table [id=ac3d135744ef4a688c5fb64c47111046]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:37:43.359122 28174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3dbd3ad800ea432c80e6827b892ebd13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:37:43.359392 28409 tablet_service.cc:1467] Processing CreateTablet for tablet 3dbd3ad800ea432c80e6827b892ebd13 (DEFAULT_TABLE table=test-table [id=ac3d135744ef4a688c5fb64c47111046]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:37:43.359490 27699 catalog_manager.cc:4841] TS 30ff76bb491e45d88aafce3ec1eb4699 (127.24.76.66:37571): tablet 893aa81595ce4270ac2ce2636a5073cd (table test-table [id=f16589be3bce438f8b9d89246a9c66f5]) successfully deleted
I20260808 11:37:43.359861 28409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3dbd3ad800ea432c80e6827b892ebd13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:37:43.364781 28748 tablet_bootstrap.cc:492] T 3dbd3ad800ea432c80e6827b892ebd13 P 3e8cee1d98e04a58a38583a87cd4acf5: Bootstrap starting.
I20260808 11:37:43.366621 28750 tablet_bootstrap.cc:492] T 3dbd3ad800ea432c80e6827b892ebd13 P 30ff76bb491e45d88aafce3ec1eb4699: Bootstrap starting.
I20260808 11:37:43.366901 28748 tablet_bootstrap.cc:654] T 3dbd3ad800ea432c80e6827b892ebd13 P 3e8cee1d98e04a58a38583a87cd4acf5: Neither blocks nor log segments found. Creating new log.
I20260808 11:37:43.367090 28747 tablet_bootstrap.cc:492] T 3dbd3ad800ea432c80e6827b892ebd13 P 7f5383ff5d894ed3b55c11b86af8af37: Bootstrap starting.
I20260808 11:37:43.368384 28750 tablet_bootstrap.cc:654] T 3dbd3ad800ea432c80e6827b892ebd13 P 30ff76bb491e45d88aafce3ec1eb4699: Neither blocks nor log segments found. Creating new log.
I20260808 11:37:43.369011 28747 tablet_bootstrap.cc:654] T 3dbd3ad800ea432c80e6827b892ebd13 P 7f5383ff5d894ed3b55c11b86af8af37: Neither blocks nor log segments found. Creating new log.
I20260808 11:37:43.369403 28748 tablet_bootstrap.cc:492] T 3dbd3ad800ea432c80e6827b892ebd13 P 3e8cee1d98e04a58a38583a87cd4acf5: No bootstrap required, opened a new log
I20260808 11:37:43.369648 28748 ts_tablet_manager.cc:1397] T 3dbd3ad800ea432c80e6827b892ebd13 P 3e8cee1d98e04a58a38583a87cd4acf5: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260808 11:37:43.370608 28750 tablet_bootstrap.cc:492] T 3dbd3ad800ea432c80e6827b892ebd13 P 30ff76bb491e45d88aafce3ec1eb4699: No bootstrap required, opened a new log
I20260808 11:37:43.370474 28748 raft_consensus.cc:348] T 3dbd3ad800ea432c80e6827b892ebd13 P 3e8cee1d98e04a58a38583a87cd4acf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } }
I20260808 11:37:43.370757 28750 ts_tablet_manager.cc:1397] T 3dbd3ad800ea432c80e6827b892ebd13 P 30ff76bb491e45d88aafce3ec1eb4699: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260808 11:37:43.370743 28748 raft_consensus.cc:374] T 3dbd3ad800ea432c80e6827b892ebd13 P 3e8cee1d98e04a58a38583a87cd4acf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:37:43.370842 28748 raft_consensus.cc:729] T 3dbd3ad800ea432c80e6827b892ebd13 P 3e8cee1d98e04a58a38583a87cd4acf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8cee1d98e04a58a38583a87cd4acf5, State: Initialized, Role: FOLLOWER
I20260808 11:37:43.371199 28748 consensus_queue.cc:260] T 3dbd3ad800ea432c80e6827b892ebd13 P 3e8cee1d98e04a58a38583a87cd4acf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } }
I20260808 11:37:43.371574 28750 raft_consensus.cc:348] T 3dbd3ad800ea432c80e6827b892ebd13 P 30ff76bb491e45d88aafce3ec1eb4699 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } }
I20260808 11:37:43.371829 28750 raft_consensus.cc:374] T 3dbd3ad800ea432c80e6827b892ebd13 P 30ff76bb491e45d88aafce3ec1eb4699 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:37:43.371932 28750 raft_consensus.cc:729] T 3dbd3ad800ea432c80e6827b892ebd13 P 30ff76bb491e45d88aafce3ec1eb4699 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30ff76bb491e45d88aafce3ec1eb4699, State: Initialized, Role: FOLLOWER
I20260808 11:37:43.372150 28750 consensus_queue.cc:260] T 3dbd3ad800ea432c80e6827b892ebd13 P 30ff76bb491e45d88aafce3ec1eb4699 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } }
I20260808 11:37:43.372551 28748 ts_tablet_manager.cc:1428] T 3dbd3ad800ea432c80e6827b892ebd13 P 3e8cee1d98e04a58a38583a87cd4acf5: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 11:37:43.372670 28750 ts_tablet_manager.cc:1428] T 3dbd3ad800ea432c80e6827b892ebd13 P 30ff76bb491e45d88aafce3ec1eb4699: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260808 11:37:43.374046 28747 tablet_bootstrap.cc:492] T 3dbd3ad800ea432c80e6827b892ebd13 P 7f5383ff5d894ed3b55c11b86af8af37: No bootstrap required, opened a new log
I20260808 11:37:43.374310 28747 ts_tablet_manager.cc:1397] T 3dbd3ad800ea432c80e6827b892ebd13 P 7f5383ff5d894ed3b55c11b86af8af37: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.005s
I20260808 11:37:43.375033 28747 raft_consensus.cc:348] T 3dbd3ad800ea432c80e6827b892ebd13 P 7f5383ff5d894ed3b55c11b86af8af37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } }
I20260808 11:37:43.375801 28747 raft_consensus.cc:374] T 3dbd3ad800ea432c80e6827b892ebd13 P 7f5383ff5d894ed3b55c11b86af8af37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:37:43.375965 28747 raft_consensus.cc:729] T 3dbd3ad800ea432c80e6827b892ebd13 P 7f5383ff5d894ed3b55c11b86af8af37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f5383ff5d894ed3b55c11b86af8af37, State: Initialized, Role: FOLLOWER
I20260808 11:37:43.376216 28747 consensus_queue.cc:260] T 3dbd3ad800ea432c80e6827b892ebd13 P 7f5383ff5d894ed3b55c11b86af8af37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } }
I20260808 11:37:43.376696 28747 ts_tablet_manager.cc:1428] T 3dbd3ad800ea432c80e6827b892ebd13 P 7f5383ff5d894ed3b55c11b86af8af37: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 11:37:43.442626 28720 raft_consensus.cc:482] T 3dbd3ad800ea432c80e6827b892ebd13 P 7f5383ff5d894ed3b55c11b86af8af37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:37:43.442863 28720 raft_consensus.cc:504] T 3dbd3ad800ea432c80e6827b892ebd13 P 7f5383ff5d894ed3b55c11b86af8af37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } }
I20260808 11:37:43.443599 28720 leader_election.cc:288] T 3dbd3ad800ea432c80e6827b892ebd13 P 7f5383ff5d894ed3b55c11b86af8af37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e8cee1d98e04a58a38583a87cd4acf5 (127.24.76.65:34007), 30ff76bb491e45d88aafce3ec1eb4699 (127.24.76.66:37571)
I20260808 11:37:43.444124 27970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dbd3ad800ea432c80e6827b892ebd13" candidate_uuid: "7f5383ff5d894ed3b55c11b86af8af37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" is_pre_election: true
I20260808 11:37:43.444190 28219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dbd3ad800ea432c80e6827b892ebd13" candidate_uuid: "7f5383ff5d894ed3b55c11b86af8af37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ff76bb491e45d88aafce3ec1eb4699" is_pre_election: true
I20260808 11:37:43.444408 28219 raft_consensus.cc:2454] T 3dbd3ad800ea432c80e6827b892ebd13 P 30ff76bb491e45d88aafce3ec1eb4699 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f5383ff5d894ed3b55c11b86af8af37 in term 0.
I20260808 11:37:43.444406 27970 raft_consensus.cc:2454] T 3dbd3ad800ea432c80e6827b892ebd13 P 3e8cee1d98e04a58a38583a87cd4acf5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f5383ff5d894ed3b55c11b86af8af37 in term 0.
I20260808 11:37:43.444830 28334 leader_election.cc:302] T 3dbd3ad800ea432c80e6827b892ebd13 P 7f5383ff5d894ed3b55c11b86af8af37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e8cee1d98e04a58a38583a87cd4acf5, 7f5383ff5d894ed3b55c11b86af8af37; no voters:
I20260808 11:37:43.445106 28720 raft_consensus.cc:2781] T 3dbd3ad800ea432c80e6827b892ebd13 P 7f5383ff5d894ed3b55c11b86af8af37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:37:43.445246 28720 raft_consensus.cc:482] T 3dbd3ad800ea432c80e6827b892ebd13 P 7f5383ff5d894ed3b55c11b86af8af37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:37:43.445329 28720 raft_consensus.cc:3037] T 3dbd3ad800ea432c80e6827b892ebd13 P 7f5383ff5d894ed3b55c11b86af8af37 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:37:43.447681 28720 raft_consensus.cc:504] T 3dbd3ad800ea432c80e6827b892ebd13 P 7f5383ff5d894ed3b55c11b86af8af37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } }
I20260808 11:37:43.448220 28720 leader_election.cc:288] T 3dbd3ad800ea432c80e6827b892ebd13 P 7f5383ff5d894ed3b55c11b86af8af37 [CANDIDATE]: Term 1 election: Requested vote from peers 3e8cee1d98e04a58a38583a87cd4acf5 (127.24.76.65:34007), 30ff76bb491e45d88aafce3ec1eb4699 (127.24.76.66:37571)
I20260808 11:37:43.448637 28219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dbd3ad800ea432c80e6827b892ebd13" candidate_uuid: "7f5383ff5d894ed3b55c11b86af8af37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ff76bb491e45d88aafce3ec1eb4699"
I20260808 11:37:43.448715 27970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3dbd3ad800ea432c80e6827b892ebd13" candidate_uuid: "7f5383ff5d894ed3b55c11b86af8af37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8cee1d98e04a58a38583a87cd4acf5"
I20260808 11:37:43.448876 28219 raft_consensus.cc:3037] T 3dbd3ad800ea432c80e6827b892ebd13 P 30ff76bb491e45d88aafce3ec1eb4699 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:37:43.448928 27970 raft_consensus.cc:3037] T 3dbd3ad800ea432c80e6827b892ebd13 P 3e8cee1d98e04a58a38583a87cd4acf5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:37:43.451310 27970 raft_consensus.cc:2454] T 3dbd3ad800ea432c80e6827b892ebd13 P 3e8cee1d98e04a58a38583a87cd4acf5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f5383ff5d894ed3b55c11b86af8af37 in term 1.
I20260808 11:37:43.451341 28219 raft_consensus.cc:2454] T 3dbd3ad800ea432c80e6827b892ebd13 P 30ff76bb491e45d88aafce3ec1eb4699 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f5383ff5d894ed3b55c11b86af8af37 in term 1.
I20260808 11:37:43.451675 28334 leader_election.cc:302] T 3dbd3ad800ea432c80e6827b892ebd13 P 7f5383ff5d894ed3b55c11b86af8af37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e8cee1d98e04a58a38583a87cd4acf5, 7f5383ff5d894ed3b55c11b86af8af37; no voters:
I20260808 11:37:43.452030 28720 raft_consensus.cc:2781] T 3dbd3ad800ea432c80e6827b892ebd13 P 7f5383ff5d894ed3b55c11b86af8af37 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:37:43.452260 28720 raft_consensus.cc:686] T 3dbd3ad800ea432c80e6827b892ebd13 P 7f5383ff5d894ed3b55c11b86af8af37 [term 1 LEADER]: Becoming Leader. State: Replica: 7f5383ff5d894ed3b55c11b86af8af37, State: Running, Role: LEADER
I20260808 11:37:43.452550 28720 consensus_queue.cc:237] T 3dbd3ad800ea432c80e6827b892ebd13 P 7f5383ff5d894ed3b55c11b86af8af37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } }
I20260808 11:37:43.455134 27729 catalog_manager.cc:5495] T 3dbd3ad800ea432c80e6827b892ebd13 P 7f5383ff5d894ed3b55c11b86af8af37 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f5383ff5d894ed3b55c11b86af8af37 (127.24.76.67). New cstate: current_term: 1 leader_uuid: "7f5383ff5d894ed3b55c11b86af8af37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f5383ff5d894ed3b55c11b86af8af37" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 39691 } health_report { overall_health: HEALTHY } } }
I20260808 11:37:43.476284 28772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:43.470040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43108 (local address 127.24.76.67:39691)
0808 11:37:43.470636 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:43.470640 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:43.470657 (+ 17us) server_negotiation.cc:407] Connection header received
0808 11:37:43.470703 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:43.470706 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:43.470762 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:43.470843 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:37:43.471416 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:43.472504 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:43.474115 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:43.474816 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:43.474927 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:43.475436 (+ 509us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:37:43.475553 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:37:43.475627 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:43.475768 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 11:37:43.475843 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":382,"threads_started":1}
I20260808 11:37:43.490554 28772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:43.484994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:53155 (local address 127.24.76.67:39691)
0808 11:37:43.485153 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:43.485160 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:43.485709 (+ 549us) server_negotiation.cc:407] Connection header received
0808 11:37:43.485877 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:43.485882 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:43.485981 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:43.486052 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:37:43.486622 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:43.487393 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:43.488908 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:43.489285 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:43.489389 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:43.489940 (+ 551us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:37:43.490051 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:37:43.490122 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:43.490226 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 11:37:43.490286 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 11:37:43.490775 28778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:43.484888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:39691 (local address 127.24.76.67:53155)
0808 11:37:43.485612 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:43.485628 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:43.485753 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:43.486079 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:43.486084 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:43.486094 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 11:37:43.486467 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:43.486478 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:43.487516 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:43.487519 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:43.488781 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:43.488794 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:43.489436 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:43.489442 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:43.489710 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:43.489761 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 11:37:43.490136 (+ 375us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 11:37:43.490145 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 11:37:43.490206 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 11:37:43.490303 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":382,"threads_started":1}
I20260808 11:37:43.497478 28219 raft_consensus.cc:1261] T 3dbd3ad800ea432c80e6827b892ebd13 P 30ff76bb491e45d88aafce3ec1eb4699 [term 1 FOLLOWER]: Refusing update from remote peer 7f5383ff5d894ed3b55c11b86af8af37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:37:43.497525 27970 raft_consensus.cc:1261] T 3dbd3ad800ea432c80e6827b892ebd13 P 3e8cee1d98e04a58a38583a87cd4acf5 [term 1 FOLLOWER]: Refusing update from remote peer 7f5383ff5d894ed3b55c11b86af8af37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:37:43.498063 28696 consensus_queue.cc:1035] T 3dbd3ad800ea432c80e6827b892ebd13 P 7f5383ff5d894ed3b55c11b86af8af37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30ff76bb491e45d88aafce3ec1eb4699" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:37:43.498404 28720 consensus_queue.cc:1035] T 3dbd3ad800ea432c80e6827b892ebd13 P 7f5383ff5d894ed3b55c11b86af8af37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e8cee1d98e04a58a38583a87cd4acf5" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 34007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:37:43.503033 28691 mvcc.cc:205] Tried to move back new op lower bound from 7316230404081385472 to 7316230403904008192. Current Snapshot: MvccSnapshot[applied={T|T < 7316230404081385472}]
W20260808 11:37:43.517256 28691 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 11:37:45.092502 27724 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:37:45.092794 27724 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:37:45.097247 27724 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1d2e375a1ba543f0ae765fab7a56ca28: Sending DeleteTablet for 3 replicas of tablet 3dbd3ad800ea432c80e6827b892ebd13
I20260808 11:37:45.098106 27944 tablet_service.cc:1514] Processing DeleteTablet for tablet 3dbd3ad800ea432c80e6827b892ebd13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:37:45 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:36300
I20260808 11:37:45.098198 28174 tablet_service.cc:1514] Processing DeleteTablet for tablet 3dbd3ad800ea432c80e6827b892ebd13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:37:45 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:60272
I20260808 11:37:45.098261 28409 tablet_service.cc:1514] Processing DeleteTablet for tablet 3dbd3ad800ea432c80e6827b892ebd13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:37:45 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:47926
I20260808 11:37:45.098814 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 27831
I20260808 11:37:45.109537 28822 tablet_replica.cc:330] T 3dbd3ad800ea432c80e6827b892ebd13 P 7f5383ff5d894ed3b55c11b86af8af37: stopping tablet replica
I20260808 11:37:45.109841 28822 raft_consensus.cc:2229] T 3dbd3ad800ea432c80e6827b892ebd13 P 7f5383ff5d894ed3b55c11b86af8af37 [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:37:45.110636 28822 raft_consensus.cc:2258] T 3dbd3ad800ea432c80e6827b892ebd13 P 7f5383ff5d894ed3b55c11b86af8af37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:37:45.110777 28814 tablet_replica.cc:330] T 3dbd3ad800ea432c80e6827b892ebd13 P 3e8cee1d98e04a58a38583a87cd4acf5: stopping tablet replica
I20260808 11:37:45.111119 28814 raft_consensus.cc:2229] T 3dbd3ad800ea432c80e6827b892ebd13 P 3e8cee1d98e04a58a38583a87cd4acf5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:37:45.111415 28819 tablet_replica.cc:330] T 3dbd3ad800ea432c80e6827b892ebd13 P 30ff76bb491e45d88aafce3ec1eb4699: stopping tablet replica
I20260808 11:37:45.111490 28814 raft_consensus.cc:2258] T 3dbd3ad800ea432c80e6827b892ebd13 P 3e8cee1d98e04a58a38583a87cd4acf5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:37:45.111616 28819 raft_consensus.cc:2229] T 3dbd3ad800ea432c80e6827b892ebd13 P 30ff76bb491e45d88aafce3ec1eb4699 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:37:45.111903 28819 raft_consensus.cc:2258] T 3dbd3ad800ea432c80e6827b892ebd13 P 30ff76bb491e45d88aafce3ec1eb4699 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:37:45.112573 28814 ts_tablet_manager.cc:1905] T 3dbd3ad800ea432c80e6827b892ebd13 P 3e8cee1d98e04a58a38583a87cd4acf5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:37:45.113065 28822 ts_tablet_manager.cc:1905] T 3dbd3ad800ea432c80e6827b892ebd13 P 7f5383ff5d894ed3b55c11b86af8af37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:37:45.113394 28819 ts_tablet_manager.cc:1905] T 3dbd3ad800ea432c80e6827b892ebd13 P 30ff76bb491e45d88aafce3ec1eb4699: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:37:45.120086 28819 ts_tablet_manager.cc:1918] T 3dbd3ad800ea432c80e6827b892ebd13 P 30ff76bb491e45d88aafce3ec1eb4699: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 11:37:45.120271 28819 log.cc:1196] T 3dbd3ad800ea432c80e6827b892ebd13 P 30ff76bb491e45d88aafce3ec1eb4699: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-1/wal/wals/3dbd3ad800ea432c80e6827b892ebd13
I20260808 11:37:45.120676 28822 ts_tablet_manager.cc:1918] T 3dbd3ad800ea432c80e6827b892ebd13 P 7f5383ff5d894ed3b55c11b86af8af37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 11:37:45.120702 28814 ts_tablet_manager.cc:1918] T 3dbd3ad800ea432c80e6827b892ebd13 P 3e8cee1d98e04a58a38583a87cd4acf5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 11:37:45.120910 28814 log.cc:1196] T 3dbd3ad800ea432c80e6827b892ebd13 P 3e8cee1d98e04a58a38583a87cd4acf5: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-0/wal/wals/3dbd3ad800ea432c80e6827b892ebd13
I20260808 11:37:45.120931 28822 log.cc:1196] T 3dbd3ad800ea432c80e6827b892ebd13 P 7f5383ff5d894ed3b55c11b86af8af37: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786189042529715-24881-0/minicluster-data/ts-2/wal/wals/3dbd3ad800ea432c80e6827b892ebd13
I20260808 11:37:45.121465 28822 ts_tablet_manager.cc:1939] T 3dbd3ad800ea432c80e6827b892ebd13 P 7f5383ff5d894ed3b55c11b86af8af37: Deleting consensus metadata
I20260808 11:37:45.121485 28814 ts_tablet_manager.cc:1939] T 3dbd3ad800ea432c80e6827b892ebd13 P 3e8cee1d98e04a58a38583a87cd4acf5: Deleting consensus metadata
Aug 08 11:37:45 dist-test-slave-tc2s krb5kdc[27653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189063, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.76.65@KRBTEST.COM
I20260808 11:37:45.122799 28819 ts_tablet_manager.cc:1939] T 3dbd3ad800ea432c80e6827b892ebd13 P 30ff76bb491e45d88aafce3ec1eb4699: Deleting consensus metadata
I20260808 11:37:45.122817 27699 catalog_manager.cc:4841] TS 7f5383ff5d894ed3b55c11b86af8af37 (127.24.76.67:39691): tablet 3dbd3ad800ea432c80e6827b892ebd13 (table test-table [id=ac3d135744ef4a688c5fb64c47111046]) successfully deleted
I20260808 11:37:45.123361 27705 catalog_manager.cc:4841] TS 3e8cee1d98e04a58a38583a87cd4acf5 (127.24.76.65:34007): tablet 3dbd3ad800ea432c80e6827b892ebd13 (table test-table [id=ac3d135744ef4a688c5fb64c47111046]) successfully deleted
I20260808 11:37:45.123941 27699 catalog_manager.cc:4841] TS 30ff76bb491e45d88aafce3ec1eb4699 (127.24.76.66:37571): tablet 3dbd3ad800ea432c80e6827b892ebd13 (table test-table [id=ac3d135744ef4a688c5fb64c47111046]) successfully deleted
I20260808 11:37:45.130249 28823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:45.099904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43798 (local address 127.24.76.65:34007)
0808 11:37:45.110297 (+ 10393us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:45.110302 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:45.110356 (+ 54us) server_negotiation.cc:407] Connection header received
0808 11:37:45.110421 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:45.110426 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:45.110511 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:45.112316 (+ 1805us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:45.113241 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:45.114552 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:45.116555 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:45.117286 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:45.118154 (+ 868us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:45.123948 (+ 5794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:45.123970 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:45.123993 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:45.124034 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:45.127656 (+ 3622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:45.128300 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:45.128307 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:45.128309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:45.128494 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:45.128768 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:45.128771 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:45.128773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:45.129144 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:45.129404 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:45.129745 (+ 341us) server_negotiation.cc:299] Negotiation successful
0808 11:37:45.129979 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10178,"thread_start_us":9957,"threads_started":1}
W20260808 11:37:45.130988 28047 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43798
W20260808 11:37:45.131608 24881 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 27831: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:37:45.131901 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 27831
I20260808 11:37:45.158353 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 28065
Aug 08 11:37:45 dist-test-slave-tc2s krb5kdc[27653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189063, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.76.66@KRBTEST.COM
I20260808 11:37:45.173180 28826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:45.159052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48132 (local address 127.24.76.66:37571)
0808 11:37:45.159680 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:45.159685 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:45.159702 (+ 17us) server_negotiation.cc:407] Connection header received
0808 11:37:45.159755 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:45.159759 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:45.159830 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:45.159927 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:45.161502 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:45.162285 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:45.163788 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:45.164204 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:45.164295 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:45.168156 (+ 3861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:45.168170 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:45.168186 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:45.168220 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:45.170997 (+ 2777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:45.171556 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:45.171564 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:45.171566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:45.171695 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:45.171962 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:45.171965 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:45.171967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:45.172323 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:45.172545 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:45.172752 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 11:37:45.172915 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":377,"threads_started":1}
W20260808 11:37:45.174106 28288 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48132
W20260808 11:37:45.174744 24881 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 28065: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:37:45.174875 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 28065
I20260808 11:37:45.204785 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 28307
Aug 08 11:37:45 dist-test-slave-tc2s krb5kdc[27653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189063, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.76.67@KRBTEST.COM
I20260808 11:37:45.219487 28827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:45.205513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43112 (local address 127.24.76.67:39691)
0808 11:37:45.205990 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:45.205994 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:45.206011 (+ 17us) server_negotiation.cc:407] Connection header received
0808 11:37:45.206063 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:45.206067 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:45.206133 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:45.206218 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:45.207716 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:45.208476 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:45.209824 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:45.210420 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:45.210521 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:45.214631 (+ 4110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:45.214648 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:45.214666 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:45.214699 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:45.217246 (+ 2547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:45.217824 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:45.217831 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:45.217833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:45.217997 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:45.218283 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:45.218286 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:45.218288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:45.218638 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:45.218857 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:45.219066 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 11:37:45.219225 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":222,"threads_started":1}
W20260808 11:37:45.220113 28532 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43112
W20260808 11:37:45.220693 24881 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 28307: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:37:45.220846 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 28307
I20260808 11:37:45.251473 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 27671
I20260808 11:37:45.252619 27782 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:37:45.370427 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 27671
2026-08-08T11:37:45Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8505 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:37:45 dist-test-slave-tc2s krb5kdc[28833](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:37:45 dist-test-slave-tc2s krb5kdc[28833](info): set up 2 sockets
Aug 08 11:37:45 dist-test-slave-tc2s krb5kdc[28833](info): commencing operation
krb5kdc: starting...
W20260808 11:37:46.568394 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:37:46 dist-test-slave-tc2s krb5kdc[28833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:37:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:37:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
I20260808 11:37:46.763441 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:38597
--webserver_interface=127.24.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:37821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:38597
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:37:47.082902 28853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:37:47.083169 28853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:37:47.083256 28853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:37:47.090282 28853 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:37:47.090381 28853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:37:47.090426 28853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:37:47.090487 28853 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:37:47.090531 28853 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:37:47.099936 28853 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:37821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:38597
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:38597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:37:47.101166 28853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:37:47.102424 28853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:37:47.113802 28863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:47.113843 28869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:47.113878 28865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:37:47.114475 28853 server_base.cc:1034] running on GCE node
I20260808 11:37:47.115156 28853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:37:47.116370 28853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:37:47.117559 28853 hybrid_clock.cc:648] HybridClock initialized: now 1786189067117507 us; error 38 us; skew 500 ppm
I20260808 11:37:47.117874 28853 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:37:47 dist-test-slave-tc2s krb5kdc[28833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189067, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:37:47.120810 28853 init.cc:377] Logged in from keytab as kudu/127.24.76.126@KRBTEST.COM (short username kudu)
I20260808 11:37:47.122299 28853 webserver.cc:460] Webserver started at http://127.24.76.126:34759/ using document root <none> and password file <none>
I20260808 11:37:47.122948 28853 fs_manager.cc:359] Metadata directory not provided
I20260808 11:37:47.123065 28853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:37:47.123351 28853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:37:47.126266 28853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "6803d015fe654f0e9dbbf4c307f04628"
format_stamp: "Formatted at 2026-08-08 11:37:47 on dist-test-slave-tc2s"
server_key: "6351fcbb7341552ebecb383bed22fdb4"
server_key_iv: "4ed80d0bd21359e0e07f140e40f4d085"
server_key_version: "encryptionkey@0"
I20260808 11:37:47.127013 28853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "6803d015fe654f0e9dbbf4c307f04628"
format_stamp: "Formatted at 2026-08-08 11:37:47 on dist-test-slave-tc2s"
server_key: "6351fcbb7341552ebecb383bed22fdb4"
server_key_iv: "4ed80d0bd21359e0e07f140e40f4d085"
server_key_version: "encryptionkey@0"
I20260808 11:37:47.132720 28853 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 11:37:47.135974 28881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:47.137157 28853 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 11:37:47.137312 28853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "6803d015fe654f0e9dbbf4c307f04628"
format_stamp: "Formatted at 2026-08-08 11:37:47 on dist-test-slave-tc2s"
server_key: "6351fcbb7341552ebecb383bed22fdb4"
server_key_iv: "4ed80d0bd21359e0e07f140e40f4d085"
server_key_version: "encryptionkey@0"
I20260808 11:37:47.137491 28853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:37:47.161340 28853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:37:47.165060 28853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:37:47.165309 28853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:37:47.182466 28853 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:38597
I20260808 11:37:47.182487 28959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:38597 every 8 connection(s)
I20260808 11:37:47.183728 28853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:37:47.187193 28960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:37:47.189258 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 28853
I20260808 11:37:47.189465 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:37:47.190065 24881 external_mini_cluster.cc:1428] Setting key 497bd691596b7f0494e11211c708d79e
I20260808 11:37:47.198514 28960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6803d015fe654f0e9dbbf4c307f04628: Bootstrap starting.
I20260808 11:37:47.202183 28960 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6803d015fe654f0e9dbbf4c307f04628: Neither blocks nor log segments found. Creating new log.
I20260808 11:37:47.203418 28960 log.cc:824] T 00000000000000000000000000000000 P 6803d015fe654f0e9dbbf4c307f04628: Log is configured to *not* fsync() on all Append() calls
Aug 08 11:37:47 dist-test-slave-tc2s krb5kdc[28833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:37:47.206511 28960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6803d015fe654f0e9dbbf4c307f04628: No bootstrap required, opened a new log
I20260808 11:37:47.212376 28966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:47.191911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39616 (local address 127.24.76.126:38597)
0808 11:37:47.192673 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:47.192684 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:47.192724 (+ 40us) server_negotiation.cc:407] Connection header received
0808 11:37:47.194398 (+ 1674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:47.194419 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:47.194761 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:47.195083 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:47.196820 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:47.198159 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:47.199820 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:47.200549 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:47.200669 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:47.205465 (+ 4796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:47.205473 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:47.205487 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:47.205529 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:47.208498 (+ 2969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:47.209136 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:47.209146 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:47.209151 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:47.209323 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:47.209695 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:47.209699 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:47.209701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:47.210261 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:47.210568 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:47.211019 (+ 451us) server_negotiation.cc:299] Negotiation successful
0808 11:37:47.211410 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":336,"threads_started":1}
I20260808 11:37:47.214110 28960 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6803d015fe654f0e9dbbf4c307f04628 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6803d015fe654f0e9dbbf4c307f04628" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 38597 } }
I20260808 11:37:47.214458 28960 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6803d015fe654f0e9dbbf4c307f04628 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:37:47.214551 28960 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6803d015fe654f0e9dbbf4c307f04628 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6803d015fe654f0e9dbbf4c307f04628, State: Initialized, Role: FOLLOWER
I20260808 11:37:47.215154 28960 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6803d015fe654f0e9dbbf4c307f04628 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6803d015fe654f0e9dbbf4c307f04628" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 38597 } }
I20260808 11:37:47.215407 28960 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6803d015fe654f0e9dbbf4c307f04628 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:37:47.215533 28960 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6803d015fe654f0e9dbbf4c307f04628 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:37:47.215694 28960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6803d015fe654f0e9dbbf4c307f04628 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:37:47.217710 28960 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6803d015fe654f0e9dbbf4c307f04628 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6803d015fe654f0e9dbbf4c307f04628" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 38597 } }
I20260808 11:37:47.218173 28960 leader_election.cc:302] T 00000000000000000000000000000000 P 6803d015fe654f0e9dbbf4c307f04628 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6803d015fe654f0e9dbbf4c307f04628; no voters:
I20260808 11:37:47.218750 28960 leader_election.cc:288] T 00000000000000000000000000000000 P 6803d015fe654f0e9dbbf4c307f04628 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:37:47.218865 28972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6803d015fe654f0e9dbbf4c307f04628 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:37:47.219879 28972 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6803d015fe654f0e9dbbf4c307f04628 [term 1 LEADER]: Becoming Leader. State: Replica: 6803d015fe654f0e9dbbf4c307f04628, State: Running, Role: LEADER
I20260808 11:37:47.220360 28972 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6803d015fe654f0e9dbbf4c307f04628 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6803d015fe654f0e9dbbf4c307f04628" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 38597 } }
I20260808 11:37:47.220695 28960 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6803d015fe654f0e9dbbf4c307f04628 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:37:47.223212 28975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6803d015fe654f0e9dbbf4c307f04628 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6803d015fe654f0e9dbbf4c307f04628. Latest consensus state: current_term: 1 leader_uuid: "6803d015fe654f0e9dbbf4c307f04628" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6803d015fe654f0e9dbbf4c307f04628" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 38597 } } }
I20260808 11:37:47.223241 28973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6803d015fe654f0e9dbbf4c307f04628 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6803d015fe654f0e9dbbf4c307f04628" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6803d015fe654f0e9dbbf4c307f04628" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 38597 } } }
I20260808 11:37:47.223660 28973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6803d015fe654f0e9dbbf4c307f04628 [sys.catalog]: This master's current role is: LEADER
I20260808 11:37:47.223661 28975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6803d015fe654f0e9dbbf4c307f04628 [sys.catalog]: This master's current role is: LEADER
I20260808 11:37:47.230517 28982 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:37:47.235759 28982 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:37:47.244653 28982 catalog_manager.cc:1347] Generated new cluster ID: ebd65ac98f6a4e43bf59c1729306e66f
I20260808 11:37:47.244751 28982 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:37:47.281273 28982 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:37:47.282624 28982 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:37:47.291594 28982 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6803d015fe654f0e9dbbf4c307f04628: Generated new TSK 0
I20260808 11:37:47.292327 28982 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab.
Entry for principal kudu/127.24.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab.
I20260808 11:37:47.353461 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.65:0
--local_ip_for_outbound_sockets=127.24.76.65
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:38597
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:37821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:37:47.759779 29004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:37:47.760038 29004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:37:47.760138 29004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:37:47.767064 29004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:37:47.767160 29004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:37:47.767274 29004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.65
I20260808 11:37:47.776357 29004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:37821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:38597
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.65
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:37:47.777457 29004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:37:47.778672 29004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:37:47.790979 29018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:47.790995 29015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:47.791121 29014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:37:47.791744 29004 server_base.cc:1034] running on GCE node
I20260808 11:37:47.792296 29004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:37:47.793134 29004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:37:47.794381 29004 hybrid_clock.cc:648] HybridClock initialized: now 1786189067794315 us; error 53 us; skew 500 ppm
I20260808 11:37:47.794692 29004 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:37:47 dist-test-slave-tc2s krb5kdc[28833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189067, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:37:47.797688 29004 init.cc:377] Logged in from keytab as kudu/127.24.76.65@KRBTEST.COM (short username kudu)
I20260808 11:37:47.799078 29004 webserver.cc:460] Webserver started at http://127.24.76.65:41867/ using document root <none> and password file <none>
I20260808 11:37:47.799712 29004 fs_manager.cc:359] Metadata directory not provided
I20260808 11:37:47.799827 29004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:37:47.800125 29004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:37:47.802934 29004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-0/data/instance:
uuid: "5693a8ead04f47d386c0bfaa0e6012ad"
format_stamp: "Formatted at 2026-08-08 11:37:47 on dist-test-slave-tc2s"
server_key: "fe8dbc7edc8eed2e2c48e87100653b66"
server_key_iv: "854c7e3d0c278b51b1871bc974fd7307"
server_key_version: "encryptionkey@0"
I20260808 11:37:47.803696 29004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance:
uuid: "5693a8ead04f47d386c0bfaa0e6012ad"
format_stamp: "Formatted at 2026-08-08 11:37:47 on dist-test-slave-tc2s"
server_key: "fe8dbc7edc8eed2e2c48e87100653b66"
server_key_iv: "854c7e3d0c278b51b1871bc974fd7307"
server_key_version: "encryptionkey@0"
I20260808 11:37:47.808638 29004 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 11:37:47.811892 29028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:47.813125 29004 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 11:37:47.813273 29004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-0/wal
uuid: "5693a8ead04f47d386c0bfaa0e6012ad"
format_stamp: "Formatted at 2026-08-08 11:37:47 on dist-test-slave-tc2s"
server_key: "fe8dbc7edc8eed2e2c48e87100653b66"
server_key_iv: "854c7e3d0c278b51b1871bc974fd7307"
server_key_version: "encryptionkey@0"
I20260808 11:37:47.813503 29004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:37:47.838200 29004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:37:47.842414 29004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:37:47.842685 29004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:37:47.843734 29004 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:37:47.845074 29004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:37:47.845147 29004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:47.845260 29004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:37:47.845311 29004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:47.876112 29004 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.65:34517
I20260808 11:37:47.876173 29205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.65:34517 every 8 connection(s)
I20260808 11:37:47.877305 29004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
I20260808 11:37:47.886776 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 29004
I20260808 11:37:47.887005 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance
I20260808 11:37:47.887724 24881 external_mini_cluster.cc:1428] Setting key d4a79654f6a4c7040662c25b2a4f114c
Aug 08 11:37:47 dist-test-slave-tc2s krb5kdc[28833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189067, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:37:47.896745 29208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:47.879803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:51651 (local address 127.24.76.126:38597)
0808 11:37:47.880245 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:47.880249 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:47.881179 (+ 930us) server_negotiation.cc:407] Connection header received
0808 11:37:47.883056 (+ 1877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:47.883062 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:47.883140 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:47.883235 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:47.884726 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:47.885471 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:47.887421 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:47.887834 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:47.887968 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:47.891400 (+ 3432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:47.891404 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:47.891408 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:47.891439 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:47.893789 (+ 2350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:47.894500 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:47.894505 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:47.894507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:47.894627 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:47.895038 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:47.895041 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:47.895043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:47.895319 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:47.895456 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:47.896157 (+ 701us) server_negotiation.cc:299] Negotiation successful
0808 11:37:47.896381 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":219,"threads_started":1}
I20260808 11:37:47.897682 29209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:47.880347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:38597 (local address 127.24.76.65:51651)
0808 11:37:47.881029 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:47.881067 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:47.882783 (+ 1716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:47.883478 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:47.883486 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:47.884008 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:37:47.884556 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:47.884574 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:47.885622 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:47.885626 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:47.887238 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:47.887255 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:47.888064 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:47.888073 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:47.888308 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:47.888967 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:37:47.889016 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:37:47.891196 (+ 2180us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:37:47.893945 (+ 2749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:47.893954 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:47.893969 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:47.894369 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:47.894731 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:47.894736 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:47.894740 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:47.894922 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:47.895464 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:37:47.895472 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:37:47.895782 (+ 310us) client_negotiation.cc:769] Sending connection context
0808 11:37:47.896023 (+ 241us) client_negotiation.cc:240] Negotiation successful
0808 11:37:47.896522 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":254,"threads_started":1}
I20260808 11:37:47.899242 29206 heartbeater.cc:347] Connected to a master server at 127.24.76.126:38597
I20260808 11:37:47.899523 29206 heartbeater.cc:464] Registering TS with master...
I20260808 11:37:47.900183 29206 heartbeater.cc:511] Master 127.24.76.126:38597 requested a full tablet report, sending...
I20260808 11:37:47.902271 28905 ts_manager.cc:194] Registered new tserver with Master: 5693a8ead04f47d386c0bfaa0e6012ad (127.24.76.65:34517)
I20260808 11:37:47.903702 28905 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.65@KRBTEST.COM'} at 127.24.76.65:51651
WARNING: no policy specified for kudu/127.24.76.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab.
Entry for principal kudu/127.24.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab.
I20260808 11:37:47.934212 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.66:0
--local_ip_for_outbound_sockets=127.24.76.66
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:38597
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:37821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:37:48.259374 29212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:37:48.259658 29212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:37:48.259764 29212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:37:48.267134 29212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:37:48.267227 29212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:37:48.267343 29212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.66
I20260808 11:37:48.276467 29212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:37821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:38597
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.66
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:37:48.277602 29212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:37:48.278822 29212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:37:48.291100 29220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:48.291122 29222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:48.291663 29219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:37:48.291962 29212 server_base.cc:1034] running on GCE node
I20260808 11:37:48.292510 29212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:37:48.293339 29212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:37:48.294530 29212 hybrid_clock.cc:648] HybridClock initialized: now 1786189068294491 us; error 27 us; skew 500 ppm
I20260808 11:37:48.294813 29212 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:37:48 dist-test-slave-tc2s krb5kdc[28833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189068, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:37:48.301662 29212 init.cc:377] Logged in from keytab as kudu/127.24.76.66@KRBTEST.COM (short username kudu)
I20260808 11:37:48.303100 29212 webserver.cc:460] Webserver started at http://127.24.76.66:35813/ using document root <none> and password file <none>
I20260808 11:37:48.303750 29212 fs_manager.cc:359] Metadata directory not provided
I20260808 11:37:48.303864 29212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:37:48.304176 29212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:37:48.307075 29212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-1/data/instance:
uuid: "c92f1b0e2aee41c595a4592af26ed271"
format_stamp: "Formatted at 2026-08-08 11:37:48 on dist-test-slave-tc2s"
server_key: "a821fbf7d73d02111b3090e36e73fbed"
server_key_iv: "2d15bfbbe54b662b6539fb48cc4d433e"
server_key_version: "encryptionkey@0"
I20260808 11:37:48.307844 29212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance:
uuid: "c92f1b0e2aee41c595a4592af26ed271"
format_stamp: "Formatted at 2026-08-08 11:37:48 on dist-test-slave-tc2s"
server_key: "a821fbf7d73d02111b3090e36e73fbed"
server_key_iv: "2d15bfbbe54b662b6539fb48cc4d433e"
server_key_version: "encryptionkey@0"
I20260808 11:37:48.312763 29212 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 11:37:48.316134 29235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:48.317432 29212 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 11:37:48.317579 29212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-1/wal
uuid: "c92f1b0e2aee41c595a4592af26ed271"
format_stamp: "Formatted at 2026-08-08 11:37:48 on dist-test-slave-tc2s"
server_key: "a821fbf7d73d02111b3090e36e73fbed"
server_key_iv: "2d15bfbbe54b662b6539fb48cc4d433e"
server_key_version: "encryptionkey@0"
I20260808 11:37:48.317755 29212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:37:48.330562 29212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:37:48.334497 29212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:37:48.334751 29212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:37:48.335614 29212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:37:48.336844 29212 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:37:48.336921 29212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:48.337049 29212 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:37:48.337103 29212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:48.366974 29212 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.66:33697
I20260808 11:37:48.367020 29427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.66:33697 every 8 connection(s)
I20260808 11:37:48.368177 29212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
I20260808 11:37:48.371661 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 29212
I20260808 11:37:48.371865 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance
I20260808 11:37:48.372363 24881 external_mini_cluster.cc:1428] Setting key 820bd1ddfd17283b311abac94459d1c7
Aug 08 11:37:48 dist-test-slave-tc2s krb5kdc[28833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189068, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:37:48.387579 29208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:48.370525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:49191 (local address 127.24.76.126:38597)
0808 11:37:48.370739 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:48.370745 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:48.371853 (+ 1108us) server_negotiation.cc:407] Connection header received
0808 11:37:48.374013 (+ 2160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:48.374017 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:48.374082 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:48.374183 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:48.375717 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:48.376416 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:48.377939 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:48.378361 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:48.378473 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:48.381737 (+ 3264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:48.381743 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:48.381746 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:48.381784 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:48.384708 (+ 2924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:48.385368 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:48.385374 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:48.385377 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:48.385505 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:48.385932 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:48.385937 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:48.385941 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:48.386193 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:48.386326 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:48.387005 (+ 679us) server_negotiation.cc:299] Negotiation successful
0808 11:37:48.387233 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 11:37:48.388468 29431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:48.370997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:38597 (local address 127.24.76.66:49191)
0808 11:37:48.371704 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:48.371745 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:48.373726 (+ 1981us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:48.374452 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:48.374464 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:48.374981 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:37:48.375545 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:48.375564 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:48.376569 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:48.376573 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:48.377769 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:48.377782 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:48.378575 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:48.378582 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:48.378803 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:48.379418 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:37:48.379440 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:37:48.381526 (+ 2086us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:37:48.384837 (+ 3311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:48.384846 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:48.384860 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:48.385253 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:48.385610 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:48.385615 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:48.385619 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:48.385800 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:48.386318 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:37:48.386326 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:37:48.386642 (+ 316us) client_negotiation.cc:769] Sending connection context
0808 11:37:48.386889 (+ 247us) client_negotiation.cc:240] Negotiation successful
0808 11:37:48.387380 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":297,"threads_started":1}
I20260808 11:37:48.389847 29428 heartbeater.cc:347] Connected to a master server at 127.24.76.126:38597
I20260808 11:37:48.390251 29428 heartbeater.cc:464] Registering TS with master...
I20260808 11:37:48.391132 29428 heartbeater.cc:511] Master 127.24.76.126:38597 requested a full tablet report, sending...
I20260808 11:37:48.392742 28905 ts_manager.cc:194] Registered new tserver with Master: c92f1b0e2aee41c595a4592af26ed271 (127.24.76.66:33697)
I20260808 11:37:48.393844 28905 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.66@KRBTEST.COM'} at 127.24.76.66:49191
WARNING: no policy specified for kudu/127.24.76.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab.
Entry for principal kudu/127.24.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab.
I20260808 11:37:48.418891 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.67:0
--local_ip_for_outbound_sockets=127.24.76.67
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:38597
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:37821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:37:48.735512 29438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:37:48.735759 29438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:37:48.735878 29438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:37:48.742755 29438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:37:48.742849 29438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:37:48.743003 29438 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.67
I20260808 11:37:48.752020 29438 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:37821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:38597
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.67
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:37:48.753125 29438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:37:48.754375 29438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:37:48.766687 29448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:48.766645 29449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:48.767083 29453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:37:48.767534 29438 server_base.cc:1034] running on GCE node
I20260808 11:37:48.768100 29438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:37:48.768949 29438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:37:48.770134 29438 hybrid_clock.cc:648] HybridClock initialized: now 1786189068770086 us; error 39 us; skew 500 ppm
I20260808 11:37:48.770412 29438 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:37:48 dist-test-slave-tc2s krb5kdc[28833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189068, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:37:48.773289 29438 init.cc:377] Logged in from keytab as kudu/127.24.76.67@KRBTEST.COM (short username kudu)
I20260808 11:37:48.774606 29438 webserver.cc:460] Webserver started at http://127.24.76.67:34171/ using document root <none> and password file <none>
I20260808 11:37:48.775204 29438 fs_manager.cc:359] Metadata directory not provided
I20260808 11:37:48.775302 29438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:37:48.775560 29438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:37:48.778293 29438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-2/data/instance:
uuid: "6e5755a64d9f43289378a952128b7d13"
format_stamp: "Formatted at 2026-08-08 11:37:48 on dist-test-slave-tc2s"
server_key: "623cb9112c47d21f339f2d66345ef7a1"
server_key_iv: "7a8b9fb821516f919e7affb6b9fb7fe1"
server_key_version: "encryptionkey@0"
I20260808 11:37:48.779023 29438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance:
uuid: "6e5755a64d9f43289378a952128b7d13"
format_stamp: "Formatted at 2026-08-08 11:37:48 on dist-test-slave-tc2s"
server_key: "623cb9112c47d21f339f2d66345ef7a1"
server_key_iv: "7a8b9fb821516f919e7affb6b9fb7fe1"
server_key_version: "encryptionkey@0"
I20260808 11:37:48.783732 29438 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260808 11:37:48.786870 29464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:48.788057 29438 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 11:37:48.788192 29438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-2/wal
uuid: "6e5755a64d9f43289378a952128b7d13"
format_stamp: "Formatted at 2026-08-08 11:37:48 on dist-test-slave-tc2s"
server_key: "623cb9112c47d21f339f2d66345ef7a1"
server_key_iv: "7a8b9fb821516f919e7affb6b9fb7fe1"
server_key_version: "encryptionkey@0"
I20260808 11:37:48.788362 29438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:37:48.833145 29438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:37:48.837196 29438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:37:48.837507 29438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:37:48.838532 29438 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:37:48.839881 29438 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:37:48.839953 29438 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:48.840070 29438 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:37:48.840121 29438 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:48.870095 29438 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.67:38803
I20260808 11:37:48.870136 29657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.67:38803 every 8 connection(s)
I20260808 11:37:48.871322 29438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
I20260808 11:37:48.876528 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 29438
I20260808 11:37:48.876735 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance
I20260808 11:37:48.877278 24881 external_mini_cluster.cc:1428] Setting key 4816933b066df83519b5074c1e74dd8b
Aug 08 11:37:48 dist-test-slave-tc2s krb5kdc[28833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189068, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:37:48.890794 29208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:48.873813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:48063 (local address 127.24.76.126:38597)
0808 11:37:48.874038 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:48.874044 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:48.875135 (+ 1091us) server_negotiation.cc:407] Connection header received
0808 11:37:48.877038 (+ 1903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:48.877044 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:48.877148 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:48.877237 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:48.879012 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:48.879804 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:48.881771 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:48.882218 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:48.882328 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:48.885487 (+ 3159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:48.885492 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:48.885495 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:48.885529 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:48.887951 (+ 2422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:48.888610 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:48.888614 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:48.888616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:48.888744 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:48.889144 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:48.889148 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:48.889150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:48.889403 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:48.889533 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:48.890217 (+ 684us) server_negotiation.cc:299] Negotiation successful
0808 11:37:48.890426 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 11:37:48.891726 29667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:48.874307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:38597 (local address 127.24.76.67:48063)
0808 11:37:48.874987 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:48.875023 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:48.876748 (+ 1725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:48.877503 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:48.877511 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:48.878161 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:37:48.878846 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:48.878867 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:48.879960 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:48.879965 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:48.881637 (+ 1672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:48.881651 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:48.882437 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:48.882445 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:48.882667 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:48.883265 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:37:48.883287 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:37:48.885299 (+ 2012us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:37:48.888094 (+ 2795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:48.888103 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:48.888117 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:48.888500 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:48.888848 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:48.888853 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:48.888857 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:48.889038 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:48.889532 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:37:48.889541 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:37:48.889824 (+ 283us) client_negotiation.cc:769] Sending connection context
0808 11:37:48.890099 (+ 275us) client_negotiation.cc:240] Negotiation successful
0808 11:37:48.890564 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":260,"threads_started":1}
I20260808 11:37:48.893023 29658 heartbeater.cc:347] Connected to a master server at 127.24.76.126:38597
I20260808 11:37:48.893289 29658 heartbeater.cc:464] Registering TS with master...
I20260808 11:37:48.893953 29658 heartbeater.cc:511] Master 127.24.76.126:38597 requested a full tablet report, sending...
I20260808 11:37:48.895269 28905 ts_manager.cc:194] Registered new tserver with Master: 6e5755a64d9f43289378a952128b7d13 (127.24.76.67:38803)
I20260808 11:37:48.896031 28905 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.67@KRBTEST.COM'} at 127.24.76.67:48063
I20260808 11:37:48.905145 24881 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:37:48.906463 29206 heartbeater.cc:503] Master 127.24.76.126:38597 was elected leader, sending a full tablet report...
W20260808 11:37:48.940857 29208 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:37:48.938544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39622 (local address 127.24.76.126:38597)
0808 11:37:48.938714 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:48.938718 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:48.939206 (+ 488us) server_negotiation.cc:407] Connection header received
0808 11:37:48.939378 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:48.939383 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:48.939448 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:48.939544 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:48.940620 (+ 1076us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39622: BlockingRecv error: recv got EOF from 127.0.0.1:39622 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 11:37:48.943958 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 29004
W20260808 11:37:48.946617 24881 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 29004: Not authorized: Client connection negotiation failed: client connection to 127.24.76.65:34517: server requires authentication, but client does not have Kerberos credentials available
I20260808 11:37:48.946797 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 29004
I20260808 11:37:48.981149 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 29212
W20260808 11:37:48.983784 24881 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 29212: Not authorized: Client connection negotiation failed: client connection to 127.24.76.66:33697: server requires authentication, but client does not have Kerberos credentials available
I20260808 11:37:48.983968 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 29212
I20260808 11:37:49.022028 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 29438
W20260808 11:37:49.024905 24881 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 29438: Not authorized: Client connection negotiation failed: client connection to 127.24.76.67:38803: server requires authentication, but client does not have Kerberos credentials available
I20260808 11:37:49.025066 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 29438
I20260808 11:37:49.048683 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 28853
I20260808 11:37:49.049798 28952 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:37:49.152634 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 28853
2026-08-08T11:37:49Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3775 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:37:49 dist-test-slave-tc2s krb5kdc[29691](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:37:49 dist-test-slave-tc2s krb5kdc[29691](info): set up 2 sockets
Aug 08 11:37:49 dist-test-slave-tc2s krb5kdc[29691](info): commencing operation
krb5kdc: starting...
W20260808 11:37:50.356088 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.002s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:37:50 dist-test-slave-tc2s krb5kdc[29691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:37:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:37:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
I20260808 11:37:50.557386 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:41445
--webserver_interface=127.24.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:35775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:41445
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:37:50.873931 29711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:37:50.874210 29711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:37:50.874339 29711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:37:50.881265 29711 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:37:50.881354 29711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:37:50.881420 29711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:37:50.881497 29711 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:37:50.881562 29711 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:37:50.891216 29711 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:35775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:41445
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:41445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:37:50.892437 29711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:37:50.893988 29711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:37:50.905395 29720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:50.905457 29723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:50.905427 29721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:37:50.906471 29711 server_base.cc:1034] running on GCE node
I20260808 11:37:50.907186 29711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:37:50.908425 29711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:37:50.909631 29711 hybrid_clock.cc:648] HybridClock initialized: now 1786189070909586 us; error 31 us; skew 500 ppm
I20260808 11:37:50.909987 29711 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:37:50 dist-test-slave-tc2s krb5kdc[29691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189070, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:37:50.912895 29711 init.cc:377] Logged in from keytab as kudu/127.24.76.126@KRBTEST.COM (short username kudu)
I20260808 11:37:50.914284 29711 webserver.cc:460] Webserver started at http://127.24.76.126:38947/ using document root <none> and password file <none>
I20260808 11:37:50.914902 29711 fs_manager.cc:359] Metadata directory not provided
I20260808 11:37:50.915019 29711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:37:50.915318 29711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:37:50.918272 29711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "d454b0d43f2545be8038ea188242c32a"
format_stamp: "Formatted at 2026-08-08 11:37:50 on dist-test-slave-tc2s"
server_key: "c6253bfc9010e99dab9984f48299f202"
server_key_iv: "fd516a26f09173d59817b0ad8f2e3631"
server_key_version: "encryptionkey@0"
I20260808 11:37:50.919128 29711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "d454b0d43f2545be8038ea188242c32a"
format_stamp: "Formatted at 2026-08-08 11:37:50 on dist-test-slave-tc2s"
server_key: "c6253bfc9010e99dab9984f48299f202"
server_key_iv: "fd516a26f09173d59817b0ad8f2e3631"
server_key_version: "encryptionkey@0"
I20260808 11:37:50.924060 29711 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260808 11:37:50.927430 29738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:50.928597 29711 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 11:37:50.928743 29711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "d454b0d43f2545be8038ea188242c32a"
format_stamp: "Formatted at 2026-08-08 11:37:50 on dist-test-slave-tc2s"
server_key: "c6253bfc9010e99dab9984f48299f202"
server_key_iv: "fd516a26f09173d59817b0ad8f2e3631"
server_key_version: "encryptionkey@0"
I20260808 11:37:50.928930 29711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:37:50.948912 29711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:37:50.952852 29711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:37:50.953096 29711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:37:50.970134 29711 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:41445
I20260808 11:37:50.970170 29834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:41445 every 8 connection(s)
I20260808 11:37:50.971396 29711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:37:50.974658 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 29711
I20260808 11:37:50.974913 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:37:50.975224 29835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:37:50.975773 24881 external_mini_cluster.cc:1428] Setting key ec0f11d6ba3ac3b781b3aedea8b3d828
I20260808 11:37:50.987151 29835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d454b0d43f2545be8038ea188242c32a: Bootstrap starting.
Aug 08 11:37:50 dist-test-slave-tc2s krb5kdc[29691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:37:50.990211 29835 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d454b0d43f2545be8038ea188242c32a: Neither blocks nor log segments found. Creating new log.
I20260808 11:37:50.991210 29835 log.cc:824] T 00000000000000000000000000000000 P d454b0d43f2545be8038ea188242c32a: Log is configured to *not* fsync() on all Append() calls
I20260808 11:37:50.993788 29835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d454b0d43f2545be8038ea188242c32a: No bootstrap required, opened a new log
I20260808 11:37:50.996737 29838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:50.977651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54852 (local address 127.24.76.126:41445)
0808 11:37:50.978527 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:50.978538 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:50.978579 (+ 41us) server_negotiation.cc:407] Connection header received
0808 11:37:50.980232 (+ 1653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:50.980253 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:50.980659 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:50.981020 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:50.982558 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:50.983802 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:50.985033 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:50.985720 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:50.985914 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:50.989871 (+ 3957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:50.989879 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:50.989927 (+ 48us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:50.989974 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:50.992857 (+ 2883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:50.993484 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:50.993491 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:50.993497 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:50.993656 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:50.994069 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:50.994073 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:50.994075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:50.994618 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:50.994955 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:50.995402 (+ 447us) server_negotiation.cc:299] Negotiation successful
0808 11:37:50.995768 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":429,"threads_started":1}
I20260808 11:37:51.000267 24881 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:37:51.001636 29835 raft_consensus.cc:348] T 00000000000000000000000000000000 P d454b0d43f2545be8038ea188242c32a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d454b0d43f2545be8038ea188242c32a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41445 } }
I20260808 11:37:51.001978 29835 raft_consensus.cc:374] T 00000000000000000000000000000000 P d454b0d43f2545be8038ea188242c32a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:37:51.002066 29835 raft_consensus.cc:729] T 00000000000000000000000000000000 P d454b0d43f2545be8038ea188242c32a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d454b0d43f2545be8038ea188242c32a, State: Initialized, Role: FOLLOWER
I20260808 11:37:51.002564 29835 consensus_queue.cc:260] T 00000000000000000000000000000000 P d454b0d43f2545be8038ea188242c32a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d454b0d43f2545be8038ea188242c32a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41445 } }
I20260808 11:37:51.002774 29835 raft_consensus.cc:388] T 00000000000000000000000000000000 P d454b0d43f2545be8038ea188242c32a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:37:51.002895 29835 raft_consensus.cc:482] T 00000000000000000000000000000000 P d454b0d43f2545be8038ea188242c32a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:37:51.003057 29835 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d454b0d43f2545be8038ea188242c32a [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:37:51.005054 29835 raft_consensus.cc:504] T 00000000000000000000000000000000 P d454b0d43f2545be8038ea188242c32a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d454b0d43f2545be8038ea188242c32a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41445 } }
I20260808 11:37:51.005488 29835 leader_election.cc:302] T 00000000000000000000000000000000 P d454b0d43f2545be8038ea188242c32a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d454b0d43f2545be8038ea188242c32a; no voters:
I20260808 11:37:51.006094 29835 leader_election.cc:288] T 00000000000000000000000000000000 P d454b0d43f2545be8038ea188242c32a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:37:51.006194 29841 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d454b0d43f2545be8038ea188242c32a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:37:51.008221 29835 sys_catalog.cc:564] T 00000000000000000000000000000000 P d454b0d43f2545be8038ea188242c32a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:37:51.009269 29841 raft_consensus.cc:686] T 00000000000000000000000000000000 P d454b0d43f2545be8038ea188242c32a [term 1 LEADER]: Becoming Leader. State: Replica: d454b0d43f2545be8038ea188242c32a, State: Running, Role: LEADER
I20260808 11:37:51.009971 29841 consensus_queue.cc:237] T 00000000000000000000000000000000 P d454b0d43f2545be8038ea188242c32a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d454b0d43f2545be8038ea188242c32a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41445 } }
I20260808 11:37:51.013008 29842 sys_catalog.cc:455] T 00000000000000000000000000000000 P d454b0d43f2545be8038ea188242c32a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d454b0d43f2545be8038ea188242c32a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d454b0d43f2545be8038ea188242c32a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41445 } } }
I20260808 11:37:51.013298 29843 sys_catalog.cc:455] T 00000000000000000000000000000000 P d454b0d43f2545be8038ea188242c32a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d454b0d43f2545be8038ea188242c32a. Latest consensus state: current_term: 1 leader_uuid: "d454b0d43f2545be8038ea188242c32a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d454b0d43f2545be8038ea188242c32a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41445 } } }
I20260808 11:37:51.013717 29842 sys_catalog.cc:458] T 00000000000000000000000000000000 P d454b0d43f2545be8038ea188242c32a [sys.catalog]: This master's current role is: LEADER
I20260808 11:37:51.013723 29843 sys_catalog.cc:458] T 00000000000000000000000000000000 P d454b0d43f2545be8038ea188242c32a [sys.catalog]: This master's current role is: LEADER
I20260808 11:37:51.018921 29855 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:37:51.024540 29855 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:37:51.047197 29855 catalog_manager.cc:1347] Generated new cluster ID: d305e86945574a41a7938e56ac7eeaae
I20260808 11:37:51.047323 29855 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:37:51.088662 29838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:51.079622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54862 (local address 127.24.76.126:41445)
0808 11:37:51.079783 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:51.079789 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:51.080325 (+ 536us) server_negotiation.cc:407] Connection header received
0808 11:37:51.080517 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:51.080523 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:51.080612 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:51.080736 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:51.081963 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:51.083092 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:51.084826 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:51.085496 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:51.085625 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:51.086763 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:51.086770 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:51.086775 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:37:51.087892 (+ 1117us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:51.087955 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:37:51.087995 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:51.088113 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:51.088208 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 11:37:51.088307 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 11:37:51.095592 29855 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:37:51.096899 29855 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:37:51.108711 29855 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d454b0d43f2545be8038ea188242c32a: Generated new TSK 0
I20260808 11:37:51.109689 29855 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:37:51.121312 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 29711
I20260808 11:37:51.122417 29830 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:37:51.217185 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 29711
2026-08-08T11:37:51Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2063 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:37:51 dist-test-slave-tc2s krb5kdc[29887](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:37:51 dist-test-slave-tc2s krb5kdc[29887](info): set up 2 sockets
Aug 08 11:37:51 dist-test-slave-tc2s krb5kdc[29887](info): commencing operation
krb5kdc: starting...
W20260808 11:37:52.400511 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:37:52 dist-test-slave-tc2s krb5kdc[29887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:37:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:37:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
I20260808 11:37:52.596513 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:46051
--webserver_interface=127.24.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:40153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:46051
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:37:52.913563 29907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:37:52.913853 29907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:37:52.914013 29907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:37:52.920967 29907 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:37:52.921085 29907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:37:52.921164 29907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:37:52.921245 29907 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:37:52.921307 29907 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:37:52.930727 29907 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:40153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:46051
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:46051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:37:52.931949 29907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:37:52.933187 29907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:37:52.944591 29916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:52.944543 29919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:52.944628 29917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:37:52.945485 29907 server_base.cc:1034] running on GCE node
I20260808 11:37:52.946252 29907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:37:52.947533 29907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:37:52.948726 29907 hybrid_clock.cc:648] HybridClock initialized: now 1786189072948684 us; error 33 us; skew 500 ppm
I20260808 11:37:52.949039 29907 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:37:52 dist-test-slave-tc2s krb5kdc[29887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189072, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:37:52.952045 29907 init.cc:377] Logged in from keytab as kudu/127.24.76.126@KRBTEST.COM (short username kudu)
I20260808 11:37:52.953475 29907 webserver.cc:460] Webserver started at http://127.24.76.126:35209/ using document root <none> and password file <none>
I20260808 11:37:52.954169 29907 fs_manager.cc:359] Metadata directory not provided
I20260808 11:37:52.954282 29907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:37:52.954560 29907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:37:52.957369 29907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "34b7560840224db08c0148c582600aaf"
format_stamp: "Formatted at 2026-08-08 11:37:52 on dist-test-slave-tc2s"
server_key: "3122c964ce92b6a7f59fc58f7c0beed7"
server_key_iv: "8e91fb7e98c2c12faeb9dd2b91ff5fe3"
server_key_version: "encryptionkey@0"
I20260808 11:37:52.958149 29907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "34b7560840224db08c0148c582600aaf"
format_stamp: "Formatted at 2026-08-08 11:37:52 on dist-test-slave-tc2s"
server_key: "3122c964ce92b6a7f59fc58f7c0beed7"
server_key_iv: "8e91fb7e98c2c12faeb9dd2b91ff5fe3"
server_key_version: "encryptionkey@0"
I20260808 11:37:52.963037 29907 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 11:37:52.966168 29929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:52.967417 29907 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 11:37:52.967568 29907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "34b7560840224db08c0148c582600aaf"
format_stamp: "Formatted at 2026-08-08 11:37:52 on dist-test-slave-tc2s"
server_key: "3122c964ce92b6a7f59fc58f7c0beed7"
server_key_iv: "8e91fb7e98c2c12faeb9dd2b91ff5fe3"
server_key_version: "encryptionkey@0"
I20260808 11:37:52.967737 29907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:37:52.990375 29907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:37:52.994478 29907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:37:52.994725 29907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:37:53.011837 29907 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:46051
I20260808 11:37:53.011871 30020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:46051 every 8 connection(s)
I20260808 11:37:53.013108 29907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:37:53.016614 30021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:37:53.023211 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 29907
I20260808 11:37:53.023442 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:37:53.024253 24881 external_mini_cluster.cc:1428] Setting key 1b08e34ee4b89c8ddfb5efa55621c4fd
I20260808 11:37:53.027695 30021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34b7560840224db08c0148c582600aaf: Bootstrap starting.
I20260808 11:37:53.031116 30021 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34b7560840224db08c0148c582600aaf: Neither blocks nor log segments found. Creating new log.
I20260808 11:37:53.032168 30021 log.cc:824] T 00000000000000000000000000000000 P 34b7560840224db08c0148c582600aaf: Log is configured to *not* fsync() on all Append() calls
I20260808 11:37:53.035145 30021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34b7560840224db08c0148c582600aaf: No bootstrap required, opened a new log
Aug 08 11:37:53 dist-test-slave-tc2s krb5kdc[29887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:37:53.043188 30021 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34b7560840224db08c0148c582600aaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34b7560840224db08c0148c582600aaf" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 46051 } }
I20260808 11:37:53.043526 30021 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34b7560840224db08c0148c582600aaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:37:53.043625 30021 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34b7560840224db08c0148c582600aaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34b7560840224db08c0148c582600aaf, State: Initialized, Role: FOLLOWER
I20260808 11:37:53.044133 30021 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34b7560840224db08c0148c582600aaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34b7560840224db08c0148c582600aaf" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 46051 } }
I20260808 11:37:53.044358 30021 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34b7560840224db08c0148c582600aaf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:37:53.044503 30021 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34b7560840224db08c0148c582600aaf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:37:53.044656 30021 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34b7560840224db08c0148c582600aaf [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:37:53.046418 30025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:53.026278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34828 (local address 127.24.76.126:46051)
0808 11:37:53.027014 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:53.027025 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:53.027066 (+ 41us) server_negotiation.cc:407] Connection header received
0808 11:37:53.028748 (+ 1682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:53.028771 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:53.029145 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:53.029453 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:53.031151 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:53.032511 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:53.033966 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:53.034614 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:53.034749 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:53.039186 (+ 4437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:53.039194 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:53.039207 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:53.039246 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:53.042373 (+ 3127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:53.043053 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:53.043060 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:53.043066 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:53.043234 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:53.043615 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:53.043620 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:53.043624 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:53.044250 (+ 626us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:53.044556 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:53.044975 (+ 419us) server_negotiation.cc:299] Negotiation successful
0808 11:37:53.045398 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":332,"threads_started":1}
I20260808 11:37:53.047076 30021 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34b7560840224db08c0148c582600aaf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34b7560840224db08c0148c582600aaf" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 46051 } }
I20260808 11:37:53.047559 30021 leader_election.cc:302] T 00000000000000000000000000000000 P 34b7560840224db08c0148c582600aaf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34b7560840224db08c0148c582600aaf; no voters:
I20260808 11:37:53.048092 30021 leader_election.cc:288] T 00000000000000000000000000000000 P 34b7560840224db08c0148c582600aaf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:37:53.048242 30027 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34b7560840224db08c0148c582600aaf [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:37:53.049568 30027 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34b7560840224db08c0148c582600aaf [term 1 LEADER]: Becoming Leader. State: Replica: 34b7560840224db08c0148c582600aaf, State: Running, Role: LEADER
I20260808 11:37:53.050122 30021 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34b7560840224db08c0148c582600aaf [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:37:53.050093 30027 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34b7560840224db08c0148c582600aaf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34b7560840224db08c0148c582600aaf" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 46051 } }
I20260808 11:37:53.056116 30028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34b7560840224db08c0148c582600aaf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34b7560840224db08c0148c582600aaf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34b7560840224db08c0148c582600aaf" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 46051 } } }
I20260808 11:37:53.056223 30029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34b7560840224db08c0148c582600aaf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34b7560840224db08c0148c582600aaf. Latest consensus state: current_term: 1 leader_uuid: "34b7560840224db08c0148c582600aaf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34b7560840224db08c0148c582600aaf" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 46051 } } }
I20260808 11:37:53.056533 30028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34b7560840224db08c0148c582600aaf [sys.catalog]: This master's current role is: LEADER
I20260808 11:37:53.056576 30029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34b7560840224db08c0148c582600aaf [sys.catalog]: This master's current role is: LEADER
W20260808 11:37:53.060333 30043 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 34b7560840224db08c0148c582600aaf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 11:37:53.060453 30043 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 11:37:53.060649 30045 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:37:53.062511 30045 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:37:53.070174 30045 catalog_manager.cc:1347] Generated new cluster ID: 86297a66ca6743a4b5709fb677f9d539
I20260808 11:37:53.070312 30045 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:37:53.090236 30045 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:37:53.091521 30045 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:37:53.105261 30045 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34b7560840224db08c0148c582600aaf: Generated new TSK 0
I20260808 11:37:53.106114 30045 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab.
Entry for principal kudu/127.24.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab.
I20260808 11:37:53.168972 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.65:0
--local_ip_for_outbound_sockets=127.24.76.65
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:46051
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:40153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:37:53.497857 30055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:37:53.498139 30055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:37:53.498266 30055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:37:53.505348 30055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:37:53.505445 30055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:37:53.505591 30055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.65
I20260808 11:37:53.514894 30055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:40153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:46051
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.65
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:37:53.516067 30055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:37:53.517323 30055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:37:53.529805 30064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:53.529836 30066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:53.529886 30063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:37:53.530645 30055 server_base.cc:1034] running on GCE node
I20260808 11:37:53.531229 30055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:37:53.532094 30055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:37:53.533290 30055 hybrid_clock.cc:648] HybridClock initialized: now 1786189073533244 us; error 36 us; skew 500 ppm
I20260808 11:37:53.533612 30055 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:37:53 dist-test-slave-tc2s krb5kdc[29887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189073, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:37:53.539588 30055 init.cc:377] Logged in from keytab as kudu/127.24.76.65@KRBTEST.COM (short username kudu)
I20260808 11:37:53.541042 30055 webserver.cc:460] Webserver started at http://127.24.76.65:44699/ using document root <none> and password file <none>
I20260808 11:37:53.541702 30055 fs_manager.cc:359] Metadata directory not provided
I20260808 11:37:53.541821 30055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:37:53.542145 30055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:37:53.544987 30055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-0/data/instance:
uuid: "2d4bea41a94f4fbb98372ae7605a23cc"
format_stamp: "Formatted at 2026-08-08 11:37:53 on dist-test-slave-tc2s"
server_key: "431c4c515addb7b599d588783a1d92ab"
server_key_iv: "7042a9939dc58b4901add39954c77ec8"
server_key_version: "encryptionkey@0"
I20260808 11:37:53.545765 30055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance:
uuid: "2d4bea41a94f4fbb98372ae7605a23cc"
format_stamp: "Formatted at 2026-08-08 11:37:53 on dist-test-slave-tc2s"
server_key: "431c4c515addb7b599d588783a1d92ab"
server_key_iv: "7042a9939dc58b4901add39954c77ec8"
server_key_version: "encryptionkey@0"
I20260808 11:37:53.550855 30055 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 11:37:53.554301 30080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:53.555627 30055 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 11:37:53.555778 30055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-0/wal
uuid: "2d4bea41a94f4fbb98372ae7605a23cc"
format_stamp: "Formatted at 2026-08-08 11:37:53 on dist-test-slave-tc2s"
server_key: "431c4c515addb7b599d588783a1d92ab"
server_key_iv: "7042a9939dc58b4901add39954c77ec8"
server_key_version: "encryptionkey@0"
I20260808 11:37:53.555953 30055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:37:53.585132 30055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:37:53.589226 30055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:37:53.589483 30055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:37:53.590416 30055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:37:53.591712 30055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:37:53.591785 30055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:53.591867 30055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:37:53.591912 30055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:53.622653 30055 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.65:44163
I20260808 11:37:53.622702 30259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.65:44163 every 8 connection(s)
I20260808 11:37:53.623871 30055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
I20260808 11:37:53.629120 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 30055
I20260808 11:37:53.629289 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance
I20260808 11:37:53.629941 24881 external_mini_cluster.cc:1428] Setting key 6936667b70f79d9fb3ffa2521037b881
Aug 08 11:37:53 dist-test-slave-tc2s krb5kdc[29887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189073, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:37:53.642756 30262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:53.626363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:41487 (local address 127.24.76.126:46051)
0808 11:37:53.626792 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:53.626796 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:53.627685 (+ 889us) server_negotiation.cc:407] Connection header received
0808 11:37:53.629505 (+ 1820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:53.629510 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:53.629587 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:53.629687 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:53.631292 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:53.632185 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:53.633700 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:53.634135 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:53.634246 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:53.637441 (+ 3195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:53.637453 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:53.637456 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:53.637488 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:53.639914 (+ 2426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:53.640583 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:53.640588 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:53.640590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:53.640706 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:53.641109 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:53.641113 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:53.641117 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:53.641366 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:53.641506 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:53.642169 (+ 663us) server_negotiation.cc:299] Negotiation successful
0808 11:37:53.642393 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":206,"threads_started":1}
I20260808 11:37:53.643579 30264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:53.626887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:46051 (local address 127.24.76.65:41487)
0808 11:37:53.627524 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:53.627567 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:53.629184 (+ 1617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:53.629969 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:53.629979 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:53.630497 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:37:53.631129 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:53.631148 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:53.632335 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:53.632339 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:53.633563 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:53.633576 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:53.634354 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:53.634362 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:53.634578 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:53.635192 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:37:53.635213 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:37:53.637245 (+ 2032us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:37:53.640043 (+ 2798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:53.640051 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:53.640066 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:53.640474 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:53.640808 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:53.640812 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:53.640816 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:53.641004 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:53.641488 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:37:53.641495 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:37:53.641789 (+ 294us) client_negotiation.cc:769] Sending connection context
0808 11:37:53.642051 (+ 262us) client_negotiation.cc:240] Negotiation successful
0808 11:37:53.642520 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":253,"threads_started":1}
I20260808 11:37:53.645033 30260 heartbeater.cc:347] Connected to a master server at 127.24.76.126:46051
I20260808 11:37:53.645398 30260 heartbeater.cc:464] Registering TS with master...
I20260808 11:37:53.646220 30260 heartbeater.cc:511] Master 127.24.76.126:46051 requested a full tablet report, sending...
I20260808 11:37:53.648638 29963 ts_manager.cc:194] Registered new tserver with Master: 2d4bea41a94f4fbb98372ae7605a23cc (127.24.76.65:44163)
I20260808 11:37:53.650661 29963 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.65@KRBTEST.COM'} at 127.24.76.65:41487
WARNING: no policy specified for kudu/127.24.76.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab.
Entry for principal kudu/127.24.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab.
I20260808 11:37:53.676834 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.66:0
--local_ip_for_outbound_sockets=127.24.76.66
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:46051
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:40153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:37:53.991413 30267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:37:53.991711 30267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:37:53.991851 30267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:37:53.998814 30267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:37:53.998927 30267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:37:53.999065 30267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.66
I20260808 11:37:54.008332 30267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:40153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:46051
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.66
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:37:54.009541 30267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:37:54.010807 30267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:37:54.023756 30272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:54.023857 30276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:54.024293 30274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:37:54.024559 30267 server_base.cc:1034] running on GCE node
I20260808 11:37:54.025122 30267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:37:54.026067 30267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:37:54.027288 30267 hybrid_clock.cc:648] HybridClock initialized: now 1786189074027232 us; error 44 us; skew 500 ppm
I20260808 11:37:54.027580 30267 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:37:54 dist-test-slave-tc2s krb5kdc[29887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189074, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:37:54.030552 30267 init.cc:377] Logged in from keytab as kudu/127.24.76.66@KRBTEST.COM (short username kudu)
I20260808 11:37:54.031904 30267 webserver.cc:460] Webserver started at http://127.24.76.66:33105/ using document root <none> and password file <none>
I20260808 11:37:54.032529 30267 fs_manager.cc:359] Metadata directory not provided
I20260808 11:37:54.032630 30267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:37:54.032876 30267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:37:54.035673 30267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-1/data/instance:
uuid: "bb0e84ac506b46e88cc75a455e8b8808"
format_stamp: "Formatted at 2026-08-08 11:37:54 on dist-test-slave-tc2s"
server_key: "bfb67cc57fc8a39750d517f7b02cfc2f"
server_key_iv: "4e68c25d23294ced36a37c05c152127d"
server_key_version: "encryptionkey@0"
I20260808 11:37:54.036394 30267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance:
uuid: "bb0e84ac506b46e88cc75a455e8b8808"
format_stamp: "Formatted at 2026-08-08 11:37:54 on dist-test-slave-tc2s"
server_key: "bfb67cc57fc8a39750d517f7b02cfc2f"
server_key_iv: "4e68c25d23294ced36a37c05c152127d"
server_key_version: "encryptionkey@0"
I20260808 11:37:54.041136 30267 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 11:37:54.044375 30289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:54.045570 30267 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 11:37:54.045707 30267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-1/wal
uuid: "bb0e84ac506b46e88cc75a455e8b8808"
format_stamp: "Formatted at 2026-08-08 11:37:54 on dist-test-slave-tc2s"
server_key: "bfb67cc57fc8a39750d517f7b02cfc2f"
server_key_iv: "4e68c25d23294ced36a37c05c152127d"
server_key_version: "encryptionkey@0"
I20260808 11:37:54.045873 30267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:37:54.059789 30267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:37:54.063413 30267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:37:54.063665 30267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:37:54.064518 30267 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:37:54.065716 30267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:37:54.065785 30267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:54.065932 30267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:37:54.065994 30267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260808 11:37:54.095813 30267 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.66:40041
I20260808 11:37:54.095849 30475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.66:40041 every 8 connection(s)
I20260808 11:37:54.096994 30267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
I20260808 11:37:54.103397 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 30267
I20260808 11:37:54.103590 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance
I20260808 11:37:54.104137 24881 external_mini_cluster.cc:1428] Setting key 959c56ef55e289bd7aff3ddd9a06d605
Aug 08 11:37:54 dist-test-slave-tc2s krb5kdc[29887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189074, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:37:54.116266 30262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:54.099419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:45995 (local address 127.24.76.126:46051)
0808 11:37:54.099657 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:54.099663 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:54.100759 (+ 1096us) server_negotiation.cc:407] Connection header received
0808 11:37:54.102491 (+ 1732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:54.102496 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:54.102566 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:54.102652 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:54.104472 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:54.105310 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:54.106986 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:54.107401 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:54.107509 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:54.110898 (+ 3389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:54.110904 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:54.110907 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:54.110940 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:54.113263 (+ 2323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:54.114076 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:54.114082 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:54.114085 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:54.114240 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:54.114625 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:54.114629 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:54.114633 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:54.114971 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:54.115128 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:54.115726 (+ 598us) server_negotiation.cc:299] Negotiation successful
0808 11:37:54.115951 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260808 11:37:54.117125 30485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:54.099839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:46051 (local address 127.24.76.66:45995)
0808 11:37:54.100592 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:54.100636 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:54.102247 (+ 1611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:54.102833 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:54.102840 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:54.103515 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:37:54.104287 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:54.104316 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:54.105477 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:54.105481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:54.106835 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:54.106848 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:54.107617 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:54.107624 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:54.107832 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:54.108430 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:37:54.108461 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:37:54.110684 (+ 2223us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:37:54.113396 (+ 2712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:54.113404 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:54.113417 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:54.113955 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:54.114338 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:54.114342 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:54.114344 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:54.114525 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:54.115085 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:37:54.115090 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:37:54.115368 (+ 278us) client_negotiation.cc:769] Sending connection context
0808 11:37:54.115590 (+ 222us) client_negotiation.cc:240] Negotiation successful
0808 11:37:54.116072 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":316,"threads_started":1}
I20260808 11:37:54.118535 30476 heartbeater.cc:347] Connected to a master server at 127.24.76.126:46051
I20260808 11:37:54.118799 30476 heartbeater.cc:464] Registering TS with master...
I20260808 11:37:54.119467 30476 heartbeater.cc:511] Master 127.24.76.126:46051 requested a full tablet report, sending...
I20260808 11:37:54.120889 29963 ts_manager.cc:194] Registered new tserver with Master: bb0e84ac506b46e88cc75a455e8b8808 (127.24.76.66:40041)
I20260808 11:37:54.122005 29963 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.66@KRBTEST.COM'} at 127.24.76.66:45995
WARNING: no policy specified for kudu/127.24.76.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab.
Entry for principal kudu/127.24.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab.
I20260808 11:37:54.150949 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.67:0
--local_ip_for_outbound_sockets=127.24.76.67
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:46051
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:40153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:37:54.466391 30491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:37:54.466647 30491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:37:54.466745 30491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:37:54.473558 30491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:37:54.473649 30491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:37:54.473762 30491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.67
I20260808 11:37:54.482738 30491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:40153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:46051
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.67
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:37:54.483856 30491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:37:54.485036 30491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:37:54.497283 30502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:54.497355 30505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:54.497377 30499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:37:54.498092 30491 server_base.cc:1034] running on GCE node
I20260808 11:37:54.498633 30491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:37:54.499557 30491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:37:54.500752 30491 hybrid_clock.cc:648] HybridClock initialized: now 1786189074500714 us; error 31 us; skew 500 ppm
I20260808 11:37:54.501049 30491 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:37:54 dist-test-slave-tc2s krb5kdc[29887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189074, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:37:54.504043 30491 init.cc:377] Logged in from keytab as kudu/127.24.76.67@KRBTEST.COM (short username kudu)
I20260808 11:37:54.505326 30491 webserver.cc:460] Webserver started at http://127.24.76.67:46469/ using document root <none> and password file <none>
I20260808 11:37:54.506067 30491 fs_manager.cc:359] Metadata directory not provided
I20260808 11:37:54.506196 30491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:37:54.506501 30491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:37:54.509323 30491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-2/data/instance:
uuid: "54a7e3b49a924fc4a3b70cdf7bd4add0"
format_stamp: "Formatted at 2026-08-08 11:37:54 on dist-test-slave-tc2s"
server_key: "cd2aff9fb1571c84e98b2d10bf45b13e"
server_key_iv: "27b5fe466f63ca8fd97ae9011b9f2383"
server_key_version: "encryptionkey@0"
I20260808 11:37:54.510118 30491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance:
uuid: "54a7e3b49a924fc4a3b70cdf7bd4add0"
format_stamp: "Formatted at 2026-08-08 11:37:54 on dist-test-slave-tc2s"
server_key: "cd2aff9fb1571c84e98b2d10bf45b13e"
server_key_iv: "27b5fe466f63ca8fd97ae9011b9f2383"
server_key_version: "encryptionkey@0"
I20260808 11:37:54.514925 30491 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 11:37:54.518131 30515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:54.519272 30491 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 11:37:54.519420 30491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-2/wal
uuid: "54a7e3b49a924fc4a3b70cdf7bd4add0"
format_stamp: "Formatted at 2026-08-08 11:37:54 on dist-test-slave-tc2s"
server_key: "cd2aff9fb1571c84e98b2d10bf45b13e"
server_key_iv: "27b5fe466f63ca8fd97ae9011b9f2383"
server_key_version: "encryptionkey@0"
I20260808 11:37:54.519594 30491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:37:54.534552 30491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:37:54.538607 30491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:37:54.538877 30491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:37:54.539846 30491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:37:54.541137 30491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:37:54.541216 30491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:54.541334 30491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:37:54.541407 30491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:54.571820 30491 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.67:35781
I20260808 11:37:54.571866 30695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.67:35781 every 8 connection(s)
I20260808 11:37:54.573101 30491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
I20260808 11:37:54.578224 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 30491
I20260808 11:37:54.578383 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance
I20260808 11:37:54.578960 24881 external_mini_cluster.cc:1428] Setting key e700d5b59b7d36aec3a1073a956f9b14
Aug 08 11:37:54 dist-test-slave-tc2s krb5kdc[29887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189074, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:37:54.591354 30262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:54.575517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:32803 (local address 127.24.76.126:46051)
0808 11:37:54.575697 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:54.575701 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:54.576813 (+ 1112us) server_negotiation.cc:407] Connection header received
0808 11:37:54.578617 (+ 1804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:54.578624 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:54.578711 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:54.578822 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:54.580353 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:54.581301 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:54.582975 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:54.583373 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:54.583471 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:54.586519 (+ 3048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:54.586524 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:54.586527 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:54.586560 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:54.588970 (+ 2410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:54.589556 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:54.589559 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:54.589561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:54.589678 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:54.590041 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:54.590044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:54.590046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:54.590293 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:54.590413 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:54.590954 (+ 541us) server_negotiation.cc:299] Negotiation successful
0808 11:37:54.591105 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 11:37:54.592170 30699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:54.575975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:46051 (local address 127.24.76.67:32803)
0808 11:37:54.576658 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:54.576698 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:54.578339 (+ 1641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:54.579002 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:54.579010 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:54.579565 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:37:54.580172 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:54.580198 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:54.581464 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:54.581469 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:54.582856 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:54.582869 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:54.583549 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:54.583553 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:54.583744 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:54.584299 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:37:54.584320 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:37:54.586335 (+ 2015us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:37:54.589071 (+ 2736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:54.589076 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:54.589087 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:54.589470 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:54.589750 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:54.589753 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:54.589755 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:54.589966 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:54.590382 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:37:54.590387 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:37:54.590672 (+ 285us) client_negotiation.cc:769] Sending connection context
0808 11:37:54.590869 (+ 197us) client_negotiation.cc:240] Negotiation successful
0808 11:37:54.591209 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":263,"threads_started":1}
I20260808 11:37:54.593369 30696 heartbeater.cc:347] Connected to a master server at 127.24.76.126:46051
I20260808 11:37:54.593649 30696 heartbeater.cc:464] Registering TS with master...
I20260808 11:37:54.594401 30696 heartbeater.cc:511] Master 127.24.76.126:46051 requested a full tablet report, sending...
I20260808 11:37:54.595688 29962 ts_manager.cc:194] Registered new tserver with Master: 54a7e3b49a924fc4a3b70cdf7bd4add0 (127.24.76.67:35781)
I20260808 11:37:54.596488 29962 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.67@KRBTEST.COM'} at 127.24.76.67:32803
I20260808 11:37:54.605713 24881 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 11:37:54 dist-test-slave-tc2s krb5kdc[29887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189074, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 11:37:54 dist-test-slave-tc2s krb5kdc[29887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189074, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:37:54.653775 30260 heartbeater.cc:503] Master 127.24.76.126:46051 was elected leader, sending a full tablet report...
I20260808 11:37:54.657188 30262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:54.644112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34836 (local address 127.24.76.126:46051)
0808 11:37:54.644341 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:54.644349 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:54.644682 (+ 333us) server_negotiation.cc:407] Connection header received
0808 11:37:54.644814 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:54.644818 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:54.644882 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:54.644963 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:54.646280 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:54.647000 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:54.648123 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:54.648533 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:54.648620 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:54.652436 (+ 3816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:54.652442 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:54.652445 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:54.652483 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:54.655044 (+ 2561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:54.655669 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:54.655674 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:54.655676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:54.655808 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:54.656091 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:54.656094 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:54.656096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:54.656375 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:54.656571 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:54.656748 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 11:37:54.656895 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260808 11:37:54.657644 29962 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:34836
I20260808 11:37:54.659183 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 30055
Aug 08 11:37:54 dist-test-slave-tc2s krb5kdc[29887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189074, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.24.76.65@KRBTEST.COM
I20260808 11:37:54.673638 30718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:54.660187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41220 (local address 127.24.76.65:44163)
0808 11:37:54.660751 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:54.660758 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:54.660781 (+ 23us) server_negotiation.cc:407] Connection header received
0808 11:37:54.660829 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:54.660835 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:54.661038 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:54.661152 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:54.662305 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:54.663301 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:54.664442 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:54.664970 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:54.665058 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:54.668686 (+ 3628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:54.668697 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:54.668710 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:54.668742 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:54.671358 (+ 2616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:54.671945 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:54.671962 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:54.671964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:54.672084 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:54.672406 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:54.672410 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:54.672413 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:54.672725 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:54.672940 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:54.673163 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 11:37:54.673348 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":290,"threads_started":1}
W20260808 11:37:54.674908 30253 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:41220
W20260808 11:37:54.675916 24881 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 30055: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:37:54.676110 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 30055
I20260808 11:37:54.702142 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 30267
Aug 08 11:37:54 dist-test-slave-tc2s krb5kdc[29887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189074, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.24.76.66@KRBTEST.COM
I20260808 11:37:54.716950 30719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:54.702925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53962 (local address 127.24.76.66:40041)
0808 11:37:54.703425 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:54.703432 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:54.703458 (+ 26us) server_negotiation.cc:407] Connection header received
0808 11:37:54.703518 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:54.703525 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:54.703725 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:54.703850 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:54.704996 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:54.706070 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:54.707232 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:54.707779 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:54.707881 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:54.711661 (+ 3780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:54.711673 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:54.711687 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:54.711719 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:54.714282 (+ 2563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:54.714909 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:54.714916 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:54.714918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:54.715044 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:54.715401 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:54.715404 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:54.715406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:54.715739 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:54.716040 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:54.716356 (+ 316us) server_negotiation.cc:299] Negotiation successful
0808 11:37:54.716626 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":239,"threads_started":1}
W20260808 11:37:54.718137 30464 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:53962
W20260808 11:37:54.719167 24881 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 30267: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:37:54.719293 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 30267
I20260808 11:37:54.741005 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 30491
Aug 08 11:37:54 dist-test-slave-tc2s krb5kdc[29887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189074, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.24.76.67@KRBTEST.COM
I20260808 11:37:54.756285 30721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:54.741829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46804 (local address 127.24.76.67:35781)
0808 11:37:54.742495 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:54.742502 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:54.742532 (+ 30us) server_negotiation.cc:407] Connection header received
0808 11:37:54.742595 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:54.742601 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:54.742804 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:54.742927 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:54.744179 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:54.745205 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:54.746452 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:54.747011 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:54.747104 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:54.751190 (+ 4086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:54.751203 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:54.751218 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:54.751248 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:54.753825 (+ 2577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:54.754532 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:54.754539 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:54.754541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:54.754655 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:54.754933 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:54.754936 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:54.754938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:54.755245 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:54.755474 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:54.755752 (+ 278us) server_negotiation.cc:299] Negotiation successful
0808 11:37:54.755944 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":328,"threads_started":1}
W20260808 11:37:54.757463 30691 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:46804
W20260808 11:37:54.758544 24881 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 30491: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:37:54.758687 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 30491
I20260808 11:37:54.787195 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 29907
I20260808 11:37:54.788339 30012 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:37:54.891872 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 29907
2026-08-08T11:37:54Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3677 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:37:54 dist-test-slave-tc2s krb5kdc[30726](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:37:54 dist-test-slave-tc2s krb5kdc[30726](info): set up 2 sockets
Aug 08 11:37:54 dist-test-slave-tc2s krb5kdc[30726](info): commencing operation
krb5kdc: starting...
W20260808 11:37:56.097851 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.003s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:37:56 dist-test-slave-tc2s krb5kdc[30726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:37:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:37:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
I20260808 11:37:56.296581 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:34161
--webserver_interface=127.24.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:38713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:34161
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:37:56.615967 30747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:37:56.616251 30747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:37:56.616394 30747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:37:56.623327 30747 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:37:56.623431 30747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:37:56.623508 30747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:37:56.623588 30747 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:37:56.623649 30747 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:37:56.632927 30747 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:38713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:34161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:34161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:37:56.634156 30747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:37:56.635391 30747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:37:56.646615 30755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:56.646896 30758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:56.647400 30754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:37:56.648031 30747 server_base.cc:1034] running on GCE node
I20260808 11:37:56.648775 30747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:37:56.650063 30747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:37:56.651281 30747 hybrid_clock.cc:648] HybridClock initialized: now 1786189076651224 us; error 40 us; skew 500 ppm
I20260808 11:37:56.651609 30747 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:37:56 dist-test-slave-tc2s krb5kdc[30726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189076, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:37:56.654621 30747 init.cc:377] Logged in from keytab as kudu/127.24.76.126@KRBTEST.COM (short username kudu)
I20260808 11:37:56.655987 30747 webserver.cc:460] Webserver started at http://127.24.76.126:34835/ using document root <none> and password file <none>
I20260808 11:37:56.656636 30747 fs_manager.cc:359] Metadata directory not provided
I20260808 11:37:56.656751 30747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:37:56.657037 30747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:37:56.659915 30747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "87a246088f774b4ba690c6a3a0c41847"
format_stamp: "Formatted at 2026-08-08 11:37:56 on dist-test-slave-tc2s"
server_key: "6211ecef18613e66ab6388a0f4804abf"
server_key_iv: "eca0870b43a647dd0d9b32b15e989e2d"
server_key_version: "encryptionkey@0"
I20260808 11:37:56.660677 30747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "87a246088f774b4ba690c6a3a0c41847"
format_stamp: "Formatted at 2026-08-08 11:37:56 on dist-test-slave-tc2s"
server_key: "6211ecef18613e66ab6388a0f4804abf"
server_key_iv: "eca0870b43a647dd0d9b32b15e989e2d"
server_key_version: "encryptionkey@0"
I20260808 11:37:56.665601 30747 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 11:37:56.668834 30766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:56.670096 30747 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 11:37:56.670249 30747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "87a246088f774b4ba690c6a3a0c41847"
format_stamp: "Formatted at 2026-08-08 11:37:56 on dist-test-slave-tc2s"
server_key: "6211ecef18613e66ab6388a0f4804abf"
server_key_iv: "eca0870b43a647dd0d9b32b15e989e2d"
server_key_version: "encryptionkey@0"
I20260808 11:37:56.670442 30747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:37:56.698343 30747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:37:56.702081 30747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:37:56.702350 30747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:37:56.719398 30747 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:34161
I20260808 11:37:56.719416 30849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:34161 every 8 connection(s)
I20260808 11:37:56.720677 30747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:37:56.724174 30851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:37:56.725605 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 30747
I20260808 11:37:56.725817 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:37:56.726457 24881 external_mini_cluster.cc:1428] Setting key 483bc6c5324b144c8149a28adeaa6095
I20260808 11:37:56.735647 30851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87a246088f774b4ba690c6a3a0c41847: Bootstrap starting.
I20260808 11:37:56.739281 30851 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87a246088f774b4ba690c6a3a0c41847: Neither blocks nor log segments found. Creating new log.
I20260808 11:37:56.740528 30851 log.cc:824] T 00000000000000000000000000000000 P 87a246088f774b4ba690c6a3a0c41847: Log is configured to *not* fsync() on all Append() calls
Aug 08 11:37:56 dist-test-slave-tc2s krb5kdc[30726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:37:56.743325 30851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87a246088f774b4ba690c6a3a0c41847: No bootstrap required, opened a new log
I20260808 11:37:56.748811 30854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:56.728413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58668 (local address 127.24.76.126:34161)
0808 11:37:56.729213 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:56.729227 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:56.729269 (+ 42us) server_negotiation.cc:407] Connection header received
0808 11:37:56.730970 (+ 1701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:56.730997 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:56.731367 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:56.731657 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:56.733503 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:56.734826 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:56.736567 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:56.737264 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:56.737399 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:56.742077 (+ 4678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:56.742085 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:56.742098 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:56.742137 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:56.744962 (+ 2825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:56.745606 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:56.745612 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:56.745618 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:56.745796 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:56.746176 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:56.746180 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:56.746182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:56.746711 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:56.746993 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:56.747458 (+ 465us) server_negotiation.cc:299] Negotiation successful
0808 11:37:56.747836 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":339,"threads_started":1}
I20260808 11:37:56.751269 30851 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87a246088f774b4ba690c6a3a0c41847 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87a246088f774b4ba690c6a3a0c41847" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 34161 } }
I20260808 11:37:56.751701 30851 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87a246088f774b4ba690c6a3a0c41847 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:37:56.751804 30851 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87a246088f774b4ba690c6a3a0c41847 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87a246088f774b4ba690c6a3a0c41847, State: Initialized, Role: FOLLOWER
I20260808 11:37:56.752305 30851 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87a246088f774b4ba690c6a3a0c41847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87a246088f774b4ba690c6a3a0c41847" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 34161 } }
I20260808 11:37:56.752532 30851 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87a246088f774b4ba690c6a3a0c41847 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:37:56.752655 30851 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87a246088f774b4ba690c6a3a0c41847 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:37:56.752801 30851 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87a246088f774b4ba690c6a3a0c41847 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:37:56.754889 30851 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87a246088f774b4ba690c6a3a0c41847 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87a246088f774b4ba690c6a3a0c41847" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 34161 } }
I20260808 11:37:56.755337 30851 leader_election.cc:302] T 00000000000000000000000000000000 P 87a246088f774b4ba690c6a3a0c41847 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87a246088f774b4ba690c6a3a0c41847; no voters:
I20260808 11:37:56.755861 30851 leader_election.cc:288] T 00000000000000000000000000000000 P 87a246088f774b4ba690c6a3a0c41847 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:37:56.755995 30857 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87a246088f774b4ba690c6a3a0c41847 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:37:56.756664 30857 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87a246088f774b4ba690c6a3a0c41847 [term 1 LEADER]: Becoming Leader. State: Replica: 87a246088f774b4ba690c6a3a0c41847, State: Running, Role: LEADER
I20260808 11:37:56.757149 30857 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87a246088f774b4ba690c6a3a0c41847 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87a246088f774b4ba690c6a3a0c41847" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 34161 } }
I20260808 11:37:56.757995 30851 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87a246088f774b4ba690c6a3a0c41847 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:37:56.759693 30861 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87a246088f774b4ba690c6a3a0c41847 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87a246088f774b4ba690c6a3a0c41847. Latest consensus state: current_term: 1 leader_uuid: "87a246088f774b4ba690c6a3a0c41847" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87a246088f774b4ba690c6a3a0c41847" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 34161 } } }
I20260808 11:37:56.760098 30861 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87a246088f774b4ba690c6a3a0c41847 [sys.catalog]: This master's current role is: LEADER
I20260808 11:37:56.759922 30860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87a246088f774b4ba690c6a3a0c41847 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87a246088f774b4ba690c6a3a0c41847" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87a246088f774b4ba690c6a3a0c41847" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 34161 } } }
I20260808 11:37:56.760219 30860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87a246088f774b4ba690c6a3a0c41847 [sys.catalog]: This master's current role is: LEADER
I20260808 11:37:56.761866 30869 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:37:56.766214 30869 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:37:56.774222 30869 catalog_manager.cc:1347] Generated new cluster ID: fcc099b5ca5444d28bae644efa507635
I20260808 11:37:56.774333 30869 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:37:56.796818 30869 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:37:56.798118 30869 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:37:56.808882 30869 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87a246088f774b4ba690c6a3a0c41847: Generated new TSK 0
I20260808 11:37:56.809675 30869 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab.
Entry for principal kudu/127.24.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab.
I20260808 11:37:56.865711 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.65:0
--local_ip_for_outbound_sockets=127.24.76.65
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:34161
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:38713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:37:57.189083 30893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:37:57.189419 30893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:37:57.189574 30893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:37:57.196720 30893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:37:57.196835 30893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:37:57.196987 30893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.65
I20260808 11:37:57.206779 30893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:38713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:34161
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.65
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:37:57.207973 30893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:37:57.209193 30893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:37:57.221343 30903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:57.221457 30904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:57.221478 30908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:37:57.222345 30893 server_base.cc:1034] running on GCE node
I20260808 11:37:57.222915 30893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:37:57.223767 30893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:37:57.224962 30893 hybrid_clock.cc:648] HybridClock initialized: now 1786189077224905 us; error 46 us; skew 500 ppm
I20260808 11:37:57.225242 30893 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:37:57 dist-test-slave-tc2s krb5kdc[30726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189077, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:37:57.228253 30893 init.cc:377] Logged in from keytab as kudu/127.24.76.65@KRBTEST.COM (short username kudu)
I20260808 11:37:57.229528 30893 webserver.cc:460] Webserver started at http://127.24.76.65:35367/ using document root <none> and password file <none>
I20260808 11:37:57.230212 30893 fs_manager.cc:359] Metadata directory not provided
I20260808 11:37:57.230324 30893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:37:57.230621 30893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:37:57.233355 30893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-0/data/instance:
uuid: "11c9901397d14cc695473b2e7c15f791"
format_stamp: "Formatted at 2026-08-08 11:37:57 on dist-test-slave-tc2s"
server_key: "caf9f2a3c1316d7f00fe4848a469d958"
server_key_iv: "79d7bddbc5ceb8185b91796f06fe309f"
server_key_version: "encryptionkey@0"
I20260808 11:37:57.234153 30893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance:
uuid: "11c9901397d14cc695473b2e7c15f791"
format_stamp: "Formatted at 2026-08-08 11:37:57 on dist-test-slave-tc2s"
server_key: "caf9f2a3c1316d7f00fe4848a469d958"
server_key_iv: "79d7bddbc5ceb8185b91796f06fe309f"
server_key_version: "encryptionkey@0"
I20260808 11:37:57.239027 30893 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 11:37:57.242352 30921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:57.243613 30893 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 11:37:57.243742 30893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-0/wal
uuid: "11c9901397d14cc695473b2e7c15f791"
format_stamp: "Formatted at 2026-08-08 11:37:57 on dist-test-slave-tc2s"
server_key: "caf9f2a3c1316d7f00fe4848a469d958"
server_key_iv: "79d7bddbc5ceb8185b91796f06fe309f"
server_key_version: "encryptionkey@0"
I20260808 11:37:57.243901 30893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:37:57.260759 30893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:37:57.264652 30893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:37:57.264878 30893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:37:57.265700 30893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:37:57.266950 30893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:37:57.267020 30893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:57.267108 30893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:37:57.267150 30893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:57.296207 30893 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.65:44343
I20260808 11:37:57.296252 31079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.65:44343 every 8 connection(s)
I20260808 11:37:57.297381 30893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
I20260808 11:37:57.307071 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 30893
I20260808 11:37:57.307269 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance
I20260808 11:37:57.308101 24881 external_mini_cluster.cc:1428] Setting key e0d3d889eb1b47552ad462628e43f372
Aug 08 11:37:57 dist-test-slave-tc2s krb5kdc[30726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189077, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:37:57.316543 31087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:57.299725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:51301 (local address 127.24.76.126:34161)
0808 11:37:57.300187 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:57.300194 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:57.301098 (+ 904us) server_negotiation.cc:407] Connection header received
0808 11:37:57.302917 (+ 1819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:57.302922 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:57.302997 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:57.303113 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:57.304496 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:57.305235 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:57.306879 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:57.307314 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:57.307432 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:57.311231 (+ 3799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:57.311236 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:57.311239 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:57.311268 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:57.313658 (+ 2390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:57.314381 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:57.314385 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:57.314387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:57.314502 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:57.314913 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:57.314916 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:57.314918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:57.315186 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:57.315317 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:57.315987 (+ 670us) server_negotiation.cc:299] Negotiation successful
0808 11:37:57.316201 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":214,"threads_started":1}
I20260808 11:37:57.317397 31091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:57.300231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:34161 (local address 127.24.76.65:51301)
0808 11:37:57.300929 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:57.300972 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:57.302649 (+ 1677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:57.303309 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:57.303316 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:57.303816 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:37:57.304355 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:57.304372 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:57.305372 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:57.305377 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:57.306741 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:57.306756 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:57.307565 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:57.307574 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:57.307859 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:57.308545 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:37:57.308574 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:37:57.311027 (+ 2453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:37:57.313789 (+ 2762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:57.313798 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:57.313813 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:57.314263 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:57.314603 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:57.314608 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:57.314612 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:57.314802 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:57.315309 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:37:57.315317 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:37:57.315639 (+ 322us) client_negotiation.cc:769] Sending connection context
0808 11:37:57.315865 (+ 226us) client_negotiation.cc:240] Negotiation successful
0808 11:37:57.316346 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":257,"threads_started":1}
I20260808 11:37:57.318853 31082 heartbeater.cc:347] Connected to a master server at 127.24.76.126:34161
I20260808 11:37:57.319167 31082 heartbeater.cc:464] Registering TS with master...
I20260808 11:37:57.319996 31082 heartbeater.cc:511] Master 127.24.76.126:34161 requested a full tablet report, sending...
I20260808 11:37:57.322283 30795 ts_manager.cc:194] Registered new tserver with Master: 11c9901397d14cc695473b2e7c15f791 (127.24.76.65:44343)
I20260808 11:37:57.324160 30795 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.65@KRBTEST.COM'} at 127.24.76.65:51301
WARNING: no policy specified for kudu/127.24.76.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab.
Entry for principal kudu/127.24.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab.
I20260808 11:37:57.358070 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.66:0
--local_ip_for_outbound_sockets=127.24.76.66
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:34161
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:38713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:37:57.687331 31094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:37:57.687599 31094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:37:57.687697 31094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:37:57.694629 31094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:37:57.694720 31094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:37:57.694833 31094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.66
I20260808 11:37:57.703949 31094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:38713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:34161
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.66
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:37:57.705072 31094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:37:57.706355 31094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:37:57.718694 31101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:57.718669 31102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:57.719293 31104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:37:57.719769 31094 server_base.cc:1034] running on GCE node
I20260808 11:37:57.720436 31094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:37:57.721498 31094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:37:57.722721 31094 hybrid_clock.cc:648] HybridClock initialized: now 1786189077722644 us; error 66 us; skew 500 ppm
I20260808 11:37:57.723053 31094 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:37:57 dist-test-slave-tc2s krb5kdc[30726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189077, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:37:57.726691 31094 init.cc:377] Logged in from keytab as kudu/127.24.76.66@KRBTEST.COM (short username kudu)
I20260808 11:37:57.730931 31094 webserver.cc:460] Webserver started at http://127.24.76.66:36287/ using document root <none> and password file <none>
I20260808 11:37:57.731554 31094 fs_manager.cc:359] Metadata directory not provided
I20260808 11:37:57.731658 31094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:37:57.731943 31094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:37:57.734728 31094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-1/data/instance:
uuid: "a494916d5acb42f59271a7def0fa3eb1"
format_stamp: "Formatted at 2026-08-08 11:37:57 on dist-test-slave-tc2s"
server_key: "7475a37d74f64bef28fbdd55c95cc06a"
server_key_iv: "dc21b3a492940e3905d0aac96b1d6ef1"
server_key_version: "encryptionkey@0"
I20260808 11:37:57.735455 31094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance:
uuid: "a494916d5acb42f59271a7def0fa3eb1"
format_stamp: "Formatted at 2026-08-08 11:37:57 on dist-test-slave-tc2s"
server_key: "7475a37d74f64bef28fbdd55c95cc06a"
server_key_iv: "dc21b3a492940e3905d0aac96b1d6ef1"
server_key_version: "encryptionkey@0"
I20260808 11:37:57.740343 31094 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 11:37:57.743634 31112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:57.744817 31094 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 11:37:57.744968 31094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-1/wal
uuid: "a494916d5acb42f59271a7def0fa3eb1"
format_stamp: "Formatted at 2026-08-08 11:37:57 on dist-test-slave-tc2s"
server_key: "7475a37d74f64bef28fbdd55c95cc06a"
server_key_iv: "dc21b3a492940e3905d0aac96b1d6ef1"
server_key_version: "encryptionkey@0"
I20260808 11:37:57.745165 31094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:37:57.764649 31094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:37:57.768291 31094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:37:57.768558 31094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:37:57.769419 31094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:37:57.771245 31094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:37:57.771323 31094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:57.771447 31094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:37:57.771499 31094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:57.800554 31094 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.66:42397
I20260808 11:37:57.800616 31281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.66:42397 every 8 connection(s)
I20260808 11:37:57.801744 31094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
I20260808 11:37:57.807125 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 31094
I20260808 11:37:57.807293 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance
I20260808 11:37:57.807705 24881 external_mini_cluster.cc:1428] Setting key 5e5f89575edc61c502d1f77fe376ea40
Aug 08 11:37:57 dist-test-slave-tc2s krb5kdc[30726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189077, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:37:57.821805 31087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:57.804267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:56863 (local address 127.24.76.126:34161)
0808 11:37:57.804456 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:57.804460 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:57.805638 (+ 1178us) server_negotiation.cc:407] Connection header received
0808 11:37:57.807602 (+ 1964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:57.807608 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:57.807683 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:57.807771 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:57.809475 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:57.810487 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:57.811987 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:57.812522 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:57.812640 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:57.815992 (+ 3352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:57.815998 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:57.816002 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:57.816038 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:57.818792 (+ 2754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:57.819604 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:57.819608 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:57.819610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:57.819730 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:57.820276 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:57.820279 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:57.820281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:57.820544 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:57.820652 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:57.821401 (+ 749us) server_negotiation.cc:299] Negotiation successful
0808 11:37:57.821578 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 11:37:57.823146 31287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:57.804774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:34161 (local address 127.24.76.66:56863)
0808 11:37:57.805477 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:57.805519 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:57.807313 (+ 1794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:57.808021 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:57.808029 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:57.808645 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:37:57.809296 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:57.809323 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:57.810626 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:57.810630 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:57.811835 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:57.811847 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:57.812703 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:57.812708 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:57.812890 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:57.813510 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:37:57.813536 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:37:57.815767 (+ 2231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:37:57.818924 (+ 3157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:57.818932 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:57.818954 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:57.819505 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:57.819825 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:57.819830 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:57.819833 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:57.820179 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:57.820659 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:37:57.820666 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:37:57.821054 (+ 388us) client_negotiation.cc:769] Sending connection context
0808 11:37:57.821328 (+ 274us) client_negotiation.cc:240] Negotiation successful
0808 11:37:57.821842 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":272,"threads_started":1}
I20260808 11:37:57.824554 31284 heartbeater.cc:347] Connected to a master server at 127.24.76.126:34161
I20260808 11:37:57.824849 31284 heartbeater.cc:464] Registering TS with master...
I20260808 11:37:57.825549 31284 heartbeater.cc:511] Master 127.24.76.126:34161 requested a full tablet report, sending...
I20260808 11:37:57.827011 30795 ts_manager.cc:194] Registered new tserver with Master: a494916d5acb42f59271a7def0fa3eb1 (127.24.76.66:42397)
I20260808 11:37:57.827899 30795 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.66@KRBTEST.COM'} at 127.24.76.66:56863
WARNING: no policy specified for kudu/127.24.76.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab.
Entry for principal kudu/127.24.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab.
I20260808 11:37:57.859234 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.67:0
--local_ip_for_outbound_sockets=127.24.76.67
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:34161
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:38713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:37:58.179600 31290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:37:58.179869 31290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:37:58.180003 31290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:37:58.187052 31290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:37:58.187161 31290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:37:58.187325 31290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.67
I20260808 11:37:58.196473 31290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:38713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:34161
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.67
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:37:58.197701 31290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:37:58.198967 31290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:37:58.211635 31305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:58.211550 31309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:37:58.211602 31306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:37:58.212731 31290 server_base.cc:1034] running on GCE node
I20260808 11:37:58.213321 31290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:37:58.214327 31290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:37:58.215533 31290 hybrid_clock.cc:648] HybridClock initialized: now 1786189078215473 us; error 50 us; skew 500 ppm
I20260808 11:37:58.215827 31290 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:37:58 dist-test-slave-tc2s krb5kdc[30726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189078, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:37:58.218994 31290 init.cc:377] Logged in from keytab as kudu/127.24.76.67@KRBTEST.COM (short username kudu)
I20260808 11:37:58.220469 31290 webserver.cc:460] Webserver started at http://127.24.76.67:38431/ using document root <none> and password file <none>
I20260808 11:37:58.221110 31290 fs_manager.cc:359] Metadata directory not provided
I20260808 11:37:58.221228 31290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:37:58.221534 31290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:37:58.224408 31290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-2/data/instance:
uuid: "29012dae31564743a03e7717ace4ef08"
format_stamp: "Formatted at 2026-08-08 11:37:58 on dist-test-slave-tc2s"
server_key: "4bcb23ecdf00eef5d11fdafe00a1b074"
server_key_iv: "acce5e0ba5706562394a627f30e1e9ec"
server_key_version: "encryptionkey@0"
I20260808 11:37:58.225167 31290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance:
uuid: "29012dae31564743a03e7717ace4ef08"
format_stamp: "Formatted at 2026-08-08 11:37:58 on dist-test-slave-tc2s"
server_key: "4bcb23ecdf00eef5d11fdafe00a1b074"
server_key_iv: "acce5e0ba5706562394a627f30e1e9ec"
server_key_version: "encryptionkey@0"
I20260808 11:37:58.230067 31290 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260808 11:37:58.233247 31317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:58.234576 31290 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:37:58.234726 31290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-2/wal
uuid: "29012dae31564743a03e7717ace4ef08"
format_stamp: "Formatted at 2026-08-08 11:37:58 on dist-test-slave-tc2s"
server_key: "4bcb23ecdf00eef5d11fdafe00a1b074"
server_key_iv: "acce5e0ba5706562394a627f30e1e9ec"
server_key_version: "encryptionkey@0"
I20260808 11:37:58.234905 31290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:37:58.249382 31290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:37:58.252933 31290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:37:58.253180 31290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:37:58.254091 31290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:37:58.255285 31290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:37:58.255370 31290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:58.255487 31290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:37:58.255538 31290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:37:58.285596 31290 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.67:44297
I20260808 11:37:58.285642 31479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.67:44297 every 8 connection(s)
I20260808 11:37:58.286806 31290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
I20260808 11:37:58.288482 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 31290
I20260808 11:37:58.288692 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance
I20260808 11:37:58.289239 24881 external_mini_cluster.cc:1428] Setting key 61e109c6f52ac4dffb35f0d42a8b9a5e
Aug 08 11:37:58 dist-test-slave-tc2s krb5kdc[30726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189078, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:37:58.306995 31087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:58.289196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:40151 (local address 127.24.76.126:34161)
0808 11:37:58.289374 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:58.289381 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:58.290845 (+ 1464us) server_negotiation.cc:407] Connection header received
0808 11:37:58.292914 (+ 2069us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:58.292919 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:58.292986 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:58.293066 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:58.294483 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:58.295232 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:58.297326 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:58.297715 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:58.297816 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:58.301549 (+ 3733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:58.301554 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:58.301558 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:58.301588 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:58.304515 (+ 2927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:58.305154 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:58.305157 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:58.305159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:58.305282 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:58.305613 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:58.305616 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:58.305618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:58.305868 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:58.306014 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:58.306603 (+ 589us) server_negotiation.cc:299] Negotiation successful
0808 11:37:58.306755 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 11:37:58.307857 31483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:58.289841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:34161 (local address 127.24.76.67:40151)
0808 11:37:58.290671 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0808 11:37:58.290721 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 11:37:58.292686 (+ 1965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:37:58.293245 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:37:58.293253 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:37:58.293745 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:37:58.294335 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:58.294353 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:58.295349 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:58.295353 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:58.297197 (+ 1844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:37:58.297212 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:37:58.297950 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:37:58.297958 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:37:58.298221 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:58.298887 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:37:58.298917 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:37:58.301362 (+ 2445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:37:58.304633 (+ 3271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:58.304639 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:58.304650 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:58.305062 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:58.305357 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:37:58.305360 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:37:58.305362 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:37:58.305532 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:37:58.306007 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:37:58.306013 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:37:58.306303 (+ 290us) client_negotiation.cc:769] Sending connection context
0808 11:37:58.306530 (+ 227us) client_negotiation.cc:240] Negotiation successful
0808 11:37:58.306881 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":325,"threads_started":1}
I20260808 11:37:58.309088 31481 heartbeater.cc:347] Connected to a master server at 127.24.76.126:34161
I20260808 11:37:58.309360 31481 heartbeater.cc:464] Registering TS with master...
I20260808 11:37:58.310086 31481 heartbeater.cc:511] Master 127.24.76.126:34161 requested a full tablet report, sending...
I20260808 11:37:58.311383 30795 ts_manager.cc:194] Registered new tserver with Master: 29012dae31564743a03e7717ace4ef08 (127.24.76.67:44297)
I20260808 11:37:58.312139 30795 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.67@KRBTEST.COM'} at 127.24.76.67:40151
I20260808 11:37:58.316301 24881 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:37:58.327453 31082 heartbeater.cc:503] Master 127.24.76.126:34161 was elected leader, sending a full tablet report...
Aug 08 11:37:58 dist-test-slave-tc2s krb5kdc[30726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 11:37:58 dist-test-slave-tc2s krb5kdc[30726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.65@KRBTEST.COM
I20260808 11:37:58.367733 31494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:58.353911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44568 (local address 127.24.76.65:44343)
0808 11:37:58.354516 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:58.354523 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:58.354551 (+ 28us) server_negotiation.cc:407] Connection header received
0808 11:37:58.354613 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:58.354619 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:58.354827 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:58.354955 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:58.356262 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:58.357251 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:58.358481 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:58.359072 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:58.359165 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:58.362811 (+ 3646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:58.362824 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:58.362838 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:58.362871 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:58.365359 (+ 2488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:58.365970 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:58.365977 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:58.365979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:58.366099 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:58.366409 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:58.366411 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:58.366413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:58.366739 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:58.366965 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:58.367177 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 11:37:58.367374 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":276,"threads_started":1}
Aug 08 11:37:58 dist-test-slave-tc2s krb5kdc[30726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:37:58.384182 31087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:58.371856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58684 (local address 127.24.76.126:34161)
0808 11:37:58.372057 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:58.372062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:58.372167 (+ 105us) server_negotiation.cc:407] Connection header received
0808 11:37:58.372323 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:58.372329 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:58.372397 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:58.372479 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:58.373682 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:58.374462 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:58.375719 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:58.376156 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:58.376267 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:58.379831 (+ 3564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:58.379837 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:58.379840 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:58.379872 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:58.382143 (+ 2271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:58.382671 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:58.382674 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:58.382676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:58.382790 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:58.383046 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:58.383048 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:58.383050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:58.383334 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:58.383518 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:58.383739 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 11:37:58.383893 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260808 11:37:58.384799 30795 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58684
I20260808 11:37:58.385655 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 30893
I20260808 11:37:58.396483 31494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:58.386554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44574 (local address 127.24.76.65:44343)
0808 11:37:58.386747 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:58.386753 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:58.386853 (+ 100us) server_negotiation.cc:407] Connection header received
0808 11:37:58.386984 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:58.386987 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:58.387053 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:58.387117 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:58.388164 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:58.388787 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:58.389869 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:58.390313 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:58.390404 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:58.392296 (+ 1892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:58.392300 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:58.392304 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:58.392332 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:58.394590 (+ 2258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:58.395119 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:58.395123 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:58.395125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:58.395228 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:58.395482 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:58.395485 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:58.395487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:58.395695 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:58.395860 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:58.396037 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 11:37:58.396194 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 11:37:58.397032 31075 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:37:58.479041 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 30893
I20260808 11:37:58.499879 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 31094
Aug 08 11:37:58 dist-test-slave-tc2s krb5kdc[30726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.66@KRBTEST.COM
I20260808 11:37:58.514771 31500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:58.500724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45778 (local address 127.24.76.66:42397)
0808 11:37:58.501247 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:58.501254 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:58.501280 (+ 26us) server_negotiation.cc:407] Connection header received
0808 11:37:58.501343 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:58.501350 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:58.501566 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:58.501700 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:58.502897 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:58.503895 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:58.505046 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:58.505574 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:58.505664 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:58.509607 (+ 3943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:58.509620 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:58.509635 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:58.509667 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:58.512242 (+ 2575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:58.512786 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:58.512792 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:58.512794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:58.512912 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:58.513198 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:58.513201 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:58.513203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:58.513529 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:58.513753 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:58.514239 (+ 486us) server_negotiation.cc:299] Negotiation successful
0808 11:37:58.514479 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":252,"threads_started":1}
I20260808 11:37:58.516074 31275 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:37:58.597285 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 31094
I20260808 11:37:58.621042 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 31290
Aug 08 11:37:58 dist-test-slave-tc2s krb5kdc[30726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.67@KRBTEST.COM
I20260808 11:37:58.638831 31502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:37:58.622049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38690 (local address 127.24.76.67:44297)
0808 11:37:58.622741 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0808 11:37:58.622749 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:37:58.622777 (+ 28us) server_negotiation.cc:407] Connection header received
0808 11:37:58.622839 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:37:58.622846 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:37:58.623074 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:37:58.623202 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:37:58.624372 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:58.625418 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:58.626693 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:37:58.627240 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:37:58.627334 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:37:58.633979 (+ 6645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:37:58.633996 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:37:58.634015 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:37:58.634055 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:37:58.636520 (+ 2465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:58.637116 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:58.637124 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:58.637126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:58.637248 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:37:58.637491 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:37:58.637494 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:37:58.637496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:37:58.637813 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:37:58.638093 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0808 11:37:58.638290 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 11:37:58.638525 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":365,"threads_started":1}
I20260808 11:37:58.640158 31472 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:37:58.726040 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 31290
I20260808 11:37:58.746459 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 30747
I20260808 11:37:58.747582 30845 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:37:58.851922 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 30747
2026-08-08T11:37:58Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3959 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:37:58 dist-test-slave-tc2s krb5kdc[31508](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:37:58 dist-test-slave-tc2s krb5kdc[31508](info): set up 2 sockets
Aug 08 11:37:58 dist-test-slave-tc2s krb5kdc[31508](info): commencing operation
krb5kdc: starting...
W20260808 11:38:00.045414 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.004s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:38:00 dist-test-slave-tc2s krb5kdc[31508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189080, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:38:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:38:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
I20260808 11:38:00.246691 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:34577
--webserver_interface=127.24.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:44381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:34577
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:38:00.565478 31531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:00.565778 31531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:00.565922 31531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:00.572887 31531 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:38:00.572996 31531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:00.573077 31531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:00.573155 31531 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:38:00.573220 31531 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:38:00.582719 31531 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:44381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:34577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:34577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:00.583961 31531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:00.585202 31531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:00.596282 31538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:00.596237 31542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:00.596319 31539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:00.597047 31531 server_base.cc:1034] running on GCE node
I20260808 11:38:00.597806 31531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:00.599184 31531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:00.600419 31531 hybrid_clock.cc:648] HybridClock initialized: now 1786189080600376 us; error 41 us; skew 500 ppm
I20260808 11:38:00.600750 31531 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:38:00 dist-test-slave-tc2s krb5kdc[31508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189080, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:00.603770 31531 init.cc:377] Logged in from keytab as kudu/127.24.76.126@KRBTEST.COM (short username kudu)
I20260808 11:38:00.604872 31531 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:00.604977 31531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:00.605274 31531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:00.608191 31531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "823a77169eaa400cac437f026fa98059"
format_stamp: "Formatted at 2026-08-08 11:38:00 on dist-test-slave-tc2s"
server_key: "78ec33e462f8eac8e5169fc4514c7020"
server_key_iv: "a025305588b14b95af9619ad5182fda1"
server_key_version: "encryptionkey@0"
I20260808 11:38:00.608963 31531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "823a77169eaa400cac437f026fa98059"
format_stamp: "Formatted at 2026-08-08 11:38:00 on dist-test-slave-tc2s"
server_key: "78ec33e462f8eac8e5169fc4514c7020"
server_key_iv: "a025305588b14b95af9619ad5182fda1"
server_key_version: "encryptionkey@0"
I20260808 11:38:00.613869 31531 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260808 11:38:00.617089 31553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:00.618299 31531 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 11:38:00.618479 31531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "823a77169eaa400cac437f026fa98059"
format_stamp: "Formatted at 2026-08-08 11:38:00 on dist-test-slave-tc2s"
server_key: "78ec33e462f8eac8e5169fc4514c7020"
server_key_iv: "a025305588b14b95af9619ad5182fda1"
server_key_version: "encryptionkey@0"
I20260808 11:38:00.618669 31531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:00.649228 31531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:00.653069 31531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:00.653312 31531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:00.670167 31531 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:34577
I20260808 11:38:00.670187 31628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:34577 every 8 connection(s)
I20260808 11:38:00.670971 31531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:38:00.674031 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 31531
I20260808 11:38:00.674237 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:38:00.674437 31630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:00.674914 24881 external_mini_cluster.cc:1428] Setting key 52c619ce48d2c0e2cf3cb5ee7b665a0a
I20260808 11:38:00.687774 31630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 823a77169eaa400cac437f026fa98059: Bootstrap starting.
Aug 08 11:38:00 dist-test-slave-tc2s krb5kdc[31508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189080, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:38:00.691290 31630 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 823a77169eaa400cac437f026fa98059: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:00.692288 31630 log.cc:824] T 00000000000000000000000000000000 P 823a77169eaa400cac437f026fa98059: Log is configured to *not* fsync() on all Append() calls
I20260808 11:38:00.695411 31630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 823a77169eaa400cac437f026fa98059: No bootstrap required, opened a new log
I20260808 11:38:00.698944 31634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:00.676812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48576 (local address 127.24.76.126:34577)
0808 11:38:00.677648 (+ 836us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:00.677659 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:00.677699 (+ 40us) server_negotiation.cc:407] Connection header received
0808 11:38:00.680056 (+ 2357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:00.680080 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:00.680489 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:00.680839 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:00.682725 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:00.684018 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:00.685802 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:00.686504 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:00.686633 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:00.691200 (+ 4567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:00.691208 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:00.691222 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:00.691263 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:00.694617 (+ 3354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:00.695242 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:00.695250 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:00.695257 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:00.695474 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:00.695844 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:00.695849 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:00.695853 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:00.696547 (+ 694us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:00.696867 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:00.697194 (+ 327us) server_negotiation.cc:299] Negotiation successful
0808 11:38:00.697627 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":379,"threads_started":1}
I20260808 11:38:00.703732 31630 raft_consensus.cc:348] T 00000000000000000000000000000000 P 823a77169eaa400cac437f026fa98059 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "823a77169eaa400cac437f026fa98059" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 34577 } }
I20260808 11:38:00.704047 31630 raft_consensus.cc:374] T 00000000000000000000000000000000 P 823a77169eaa400cac437f026fa98059 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:00.704121 31630 raft_consensus.cc:729] T 00000000000000000000000000000000 P 823a77169eaa400cac437f026fa98059 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 823a77169eaa400cac437f026fa98059, State: Initialized, Role: FOLLOWER
I20260808 11:38:00.704557 31630 consensus_queue.cc:260] T 00000000000000000000000000000000 P 823a77169eaa400cac437f026fa98059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "823a77169eaa400cac437f026fa98059" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 34577 } }
I20260808 11:38:00.704746 31630 raft_consensus.cc:388] T 00000000000000000000000000000000 P 823a77169eaa400cac437f026fa98059 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:38:00.704836 31630 raft_consensus.cc:482] T 00000000000000000000000000000000 P 823a77169eaa400cac437f026fa98059 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:38:00.704957 31630 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 823a77169eaa400cac437f026fa98059 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:00.706980 31630 raft_consensus.cc:504] T 00000000000000000000000000000000 P 823a77169eaa400cac437f026fa98059 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "823a77169eaa400cac437f026fa98059" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 34577 } }
I20260808 11:38:00.707434 31630 leader_election.cc:302] T 00000000000000000000000000000000 P 823a77169eaa400cac437f026fa98059 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 823a77169eaa400cac437f026fa98059; no voters:
I20260808 11:38:00.707962 31630 leader_election.cc:288] T 00000000000000000000000000000000 P 823a77169eaa400cac437f026fa98059 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:38:00.708072 31637 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 823a77169eaa400cac437f026fa98059 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:38:00.708693 31637 raft_consensus.cc:686] T 00000000000000000000000000000000 P 823a77169eaa400cac437f026fa98059 [term 1 LEADER]: Becoming Leader. State: Replica: 823a77169eaa400cac437f026fa98059, State: Running, Role: LEADER
I20260808 11:38:00.709182 31637 consensus_queue.cc:237] T 00000000000000000000000000000000 P 823a77169eaa400cac437f026fa98059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "823a77169eaa400cac437f026fa98059" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 34577 } }
I20260808 11:38:00.709820 31630 sys_catalog.cc:564] T 00000000000000000000000000000000 P 823a77169eaa400cac437f026fa98059 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:38:00.712311 31639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 823a77169eaa400cac437f026fa98059 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 823a77169eaa400cac437f026fa98059. Latest consensus state: current_term: 1 leader_uuid: "823a77169eaa400cac437f026fa98059" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "823a77169eaa400cac437f026fa98059" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 34577 } } }
I20260808 11:38:00.712356 31638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 823a77169eaa400cac437f026fa98059 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "823a77169eaa400cac437f026fa98059" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "823a77169eaa400cac437f026fa98059" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 34577 } } }
I20260808 11:38:00.712740 31639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 823a77169eaa400cac437f026fa98059 [sys.catalog]: This master's current role is: LEADER
I20260808 11:38:00.712790 31638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 823a77169eaa400cac437f026fa98059 [sys.catalog]: This master's current role is: LEADER
I20260808 11:38:00.714581 31645 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:38:00.720044 31645 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:38:00.727538 31645 catalog_manager.cc:1347] Generated new cluster ID: c081a4f6b1034058bf0fa0ab3d6089ce
I20260808 11:38:00.727640 31645 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:38:00.745072 31645 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:38:00.746330 31645 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:38:00.750780 31645 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 823a77169eaa400cac437f026fa98059: Generated new TSK 0
I20260808 11:38:00.751464 31645 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab.
Entry for principal kudu/127.24.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab.
I20260808 11:38:00.801342 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.65:0
--local_ip_for_outbound_sockets=127.24.76.65
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:34577
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:44381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:38:01.122443 31668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:01.122731 31668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:01.122866 31668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:01.129739 31668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:01.129830 31668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:01.129979 31668 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.65
I20260808 11:38:01.139061 31668 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:44381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:34577
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.65
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:01.140141 31668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:01.141360 31668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:01.153227 31678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:01.153343 31681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:01.153383 31683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:01.154647 31668 server_base.cc:1034] running on GCE node
I20260808 11:38:01.155202 31668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:01.156076 31668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:01.157284 31668 hybrid_clock.cc:648] HybridClock initialized: now 1786189081157237 us; error 43 us; skew 500 ppm
I20260808 11:38:01.157588 31668 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:38:01 dist-test-slave-tc2s krb5kdc[31508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189081, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:01.160591 31668 init.cc:377] Logged in from keytab as kudu/127.24.76.65@KRBTEST.COM (short username kudu)
I20260808 11:38:01.161737 31668 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:01.161837 31668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:01.162168 31668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:01.165002 31668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-0/data/instance:
uuid: "439585dddb4040f9809bcdfd0bbaa142"
format_stamp: "Formatted at 2026-08-08 11:38:01 on dist-test-slave-tc2s"
server_key: "d4da16e7bd00cc27ea262cc1fb00d050"
server_key_iv: "2f93ed8334bb1bba29d373730f687be7"
server_key_version: "encryptionkey@0"
I20260808 11:38:01.165788 31668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance:
uuid: "439585dddb4040f9809bcdfd0bbaa142"
format_stamp: "Formatted at 2026-08-08 11:38:01 on dist-test-slave-tc2s"
server_key: "d4da16e7bd00cc27ea262cc1fb00d050"
server_key_iv: "2f93ed8334bb1bba29d373730f687be7"
server_key_version: "encryptionkey@0"
I20260808 11:38:01.170507 31668 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 11:38:01.173735 31689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:01.174958 31668 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 11:38:01.175107 31668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-0/wal
uuid: "439585dddb4040f9809bcdfd0bbaa142"
format_stamp: "Formatted at 2026-08-08 11:38:01 on dist-test-slave-tc2s"
server_key: "d4da16e7bd00cc27ea262cc1fb00d050"
server_key_iv: "2f93ed8334bb1bba29d373730f687be7"
server_key_version: "encryptionkey@0"
I20260808 11:38:01.175309 31668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:01.188669 31668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:01.192214 31668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:01.192446 31668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:01.193310 31668 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:38:01.194621 31668 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:38:01.194715 31668 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:01.194814 31668 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:38:01.194883 31668 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:01.224478 31668 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.65:44923
I20260808 11:38:01.224551 31864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.65:44923 every 8 connection(s)
I20260808 11:38:01.225283 31668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
I20260808 11:38:01.232268 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 31668
I20260808 11:38:01.232458 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance
I20260808 11:38:01.233075 24881 external_mini_cluster.cc:1428] Setting key fef03ccd972ae60dc00c06ebd12afa7a
Aug 08 11:38:01 dist-test-slave-tc2s krb5kdc[31508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189081, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:38:01.245034 31869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:01.227547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:41945 (local address 127.24.76.126:34577)
0808 11:38:01.227955 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:01.227960 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:01.228783 (+ 823us) server_negotiation.cc:407] Connection header received
0808 11:38:01.230638 (+ 1855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:01.230644 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:01.230722 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:01.230822 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:01.232265 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:01.233057 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:01.234736 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:01.235248 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:01.235373 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:01.238873 (+ 3500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:01.238880 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:01.238885 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:01.238923 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:01.242073 (+ 3150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:01.242866 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:01.242871 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:01.242874 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:01.243019 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:01.243436 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:01.243440 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:01.243444 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:01.243725 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:01.243835 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:01.244598 (+ 763us) server_negotiation.cc:299] Negotiation successful
0808 11:38:01.244764 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":218,"threads_started":1}
I20260808 11:38:01.246069 31871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:01.227949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:34577 (local address 127.24.76.65:41945)
0808 11:38:01.228624 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:01.228673 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:01.230379 (+ 1706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:01.231061 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:01.231069 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:01.231564 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:01.232099 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:01.232116 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:01.233204 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:01.233208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:01.234602 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:01.234616 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:01.235445 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:01.235450 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:01.235632 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:01.236232 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:01.236258 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:01.238630 (+ 2372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:01.242205 (+ 3575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:01.242213 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:01.242227 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:01.242747 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:01.243122 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:01.243126 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:01.243129 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:01.243340 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:01.243849 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:01.243857 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:01.244241 (+ 384us) client_negotiation.cc:769] Sending connection context
0808 11:38:01.244538 (+ 297us) client_negotiation.cc:240] Negotiation successful
0808 11:38:01.245019 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":260,"threads_started":1}
I20260808 11:38:01.247452 31866 heartbeater.cc:347] Connected to a master server at 127.24.76.126:34577
I20260808 11:38:01.247766 31866 heartbeater.cc:464] Registering TS with master...
I20260808 11:38:01.248514 31866 heartbeater.cc:511] Master 127.24.76.126:34577 requested a full tablet report, sending...
I20260808 11:38:01.250716 31578 ts_manager.cc:194] Registered new tserver with Master: 439585dddb4040f9809bcdfd0bbaa142 (127.24.76.65:44923)
I20260808 11:38:01.252173 31578 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.65@KRBTEST.COM'} at 127.24.76.65:41945
WARNING: no policy specified for kudu/127.24.76.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab.
Entry for principal kudu/127.24.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab.
I20260808 11:38:01.283197 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.66:0
--local_ip_for_outbound_sockets=127.24.76.66
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:34577
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:44381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:38:01.603751 31875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:01.604018 31875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:01.604152 31875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:01.611109 31875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:01.611218 31875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:01.611399 31875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.66
I20260808 11:38:01.620638 31875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:44381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:34577
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.66
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:01.621942 31875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:01.623288 31875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:01.635417 31881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:01.635478 31882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:01.635571 31885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:01.636451 31875 server_base.cc:1034] running on GCE node
I20260808 11:38:01.637032 31875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:01.637945 31875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:01.639153 31875 hybrid_clock.cc:648] HybridClock initialized: now 1786189081639099 us; error 46 us; skew 500 ppm
I20260808 11:38:01.639446 31875 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:38:01 dist-test-slave-tc2s krb5kdc[31508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189081, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:01.642403 31875 init.cc:377] Logged in from keytab as kudu/127.24.76.66@KRBTEST.COM (short username kudu)
I20260808 11:38:01.643461 31875 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:01.643566 31875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:01.643863 31875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:01.646729 31875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-1/data/instance:
uuid: "e2ca1793304a406b99da5dff38ebf56b"
format_stamp: "Formatted at 2026-08-08 11:38:01 on dist-test-slave-tc2s"
server_key: "f70b8eea7acf4fc2bb523e8ad7ab2749"
server_key_iv: "4536d602f6d569ae26a06ebf60f7c6b2"
server_key_version: "encryptionkey@0"
I20260808 11:38:01.647505 31875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance:
uuid: "e2ca1793304a406b99da5dff38ebf56b"
format_stamp: "Formatted at 2026-08-08 11:38:01 on dist-test-slave-tc2s"
server_key: "f70b8eea7acf4fc2bb523e8ad7ab2749"
server_key_iv: "4536d602f6d569ae26a06ebf60f7c6b2"
server_key_version: "encryptionkey@0"
I20260808 11:38:01.652246 31875 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 11:38:01.655578 31893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:01.656813 31875 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 11:38:01.656965 31875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-1/wal
uuid: "e2ca1793304a406b99da5dff38ebf56b"
format_stamp: "Formatted at 2026-08-08 11:38:01 on dist-test-slave-tc2s"
server_key: "f70b8eea7acf4fc2bb523e8ad7ab2749"
server_key_iv: "4536d602f6d569ae26a06ebf60f7c6b2"
server_key_version: "encryptionkey@0"
I20260808 11:38:01.657178 31875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:01.681022 31875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:01.684801 31875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:01.685055 31875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:01.685999 31875 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:38:01.687318 31875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:38:01.687415 31875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:01.687521 31875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:38:01.687592 31875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:01.719385 31875 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.66:41513
I20260808 11:38:01.719437 32081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.66:41513 every 8 connection(s)
I20260808 11:38:01.720182 31875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
I20260808 11:38:01.722220 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 31875
I20260808 11:38:01.722411 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance
I20260808 11:38:01.722980 24881 external_mini_cluster.cc:1428] Setting key dd21a4c050e565e8917814a0fd810d63
Aug 08 11:38:01 dist-test-slave-tc2s krb5kdc[31508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189081, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:38:01.739909 31869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:01.722558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:54467 (local address 127.24.76.126:34577)
0808 11:38:01.722770 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:01.722777 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:01.723934 (+ 1157us) server_negotiation.cc:407] Connection header received
0808 11:38:01.725793 (+ 1859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:01.725798 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:01.725874 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:01.726008 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:01.727466 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:01.728399 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:01.729931 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:01.730525 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:01.730641 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:01.734095 (+ 3454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:01.734102 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:01.734107 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:01.734146 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:01.737047 (+ 2901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:01.737705 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:01.737710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:01.737713 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:01.737869 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:01.738271 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:01.738276 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:01.738278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:01.738613 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:01.738744 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:01.739366 (+ 622us) server_negotiation.cc:299] Negotiation successful
0808 11:38:01.739595 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260808 11:38:01.740654 32084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:01.723007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:34577 (local address 127.24.76.66:54467)
0808 11:38:01.723787 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:01.723833 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:01.725544 (+ 1711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:01.726183 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:01.726190 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:01.726754 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:01.727302 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:01.727319 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:01.728514 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:01.728517 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:01.729739 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:01.729752 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:01.730707 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:01.730711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:01.730900 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:01.731533 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:01.731553 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:01.733852 (+ 2299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:01.737170 (+ 3318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:01.737177 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:01.737189 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:01.737589 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:01.737979 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:01.737996 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:01.737998 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:01.738169 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:01.738715 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:01.738721 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:01.739015 (+ 294us) client_negotiation.cc:769] Sending connection context
0808 11:38:01.739236 (+ 221us) client_negotiation.cc:240] Negotiation successful
0808 11:38:01.739593 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":325,"threads_started":1}
I20260808 11:38:01.742002 32082 heartbeater.cc:347] Connected to a master server at 127.24.76.126:34577
I20260808 11:38:01.742314 32082 heartbeater.cc:464] Registering TS with master...
I20260808 11:38:01.743079 32082 heartbeater.cc:511] Master 127.24.76.126:34577 requested a full tablet report, sending...
I20260808 11:38:01.744633 31578 ts_manager.cc:194] Registered new tserver with Master: e2ca1793304a406b99da5dff38ebf56b (127.24.76.66:41513)
I20260808 11:38:01.745404 31578 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.66@KRBTEST.COM'} at 127.24.76.66:54467
WARNING: no policy specified for kudu/127.24.76.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab.
Entry for principal kudu/127.24.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab.
I20260808 11:38:01.772693 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.67:0
--local_ip_for_outbound_sockets=127.24.76.67
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:34577
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:44381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:38:02.088625 32087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:02.088883 32087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:02.089008 32087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:02.095988 32087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:02.096091 32087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:02.096257 32087 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.67
I20260808 11:38:02.105492 32087 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:44381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:34577
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.67
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:02.106729 32087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:02.108013 32087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:02.120357 32093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:02.120352 32094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:02.120519 32097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:02.121233 32087 server_base.cc:1034] running on GCE node
I20260808 11:38:02.121788 32087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:02.122659 32087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:02.123847 32087 hybrid_clock.cc:648] HybridClock initialized: now 1786189082123801 us; error 42 us; skew 500 ppm
I20260808 11:38:02.124159 32087 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:38:02 dist-test-slave-tc2s krb5kdc[31508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189082, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:02.127120 32087 init.cc:377] Logged in from keytab as kudu/127.24.76.67@KRBTEST.COM (short username kudu)
I20260808 11:38:02.128129 32087 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:02.128242 32087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:02.128552 32087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:02.131374 32087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-2/data/instance:
uuid: "09a72b38ec7245b382dae73df7f254fb"
format_stamp: "Formatted at 2026-08-08 11:38:02 on dist-test-slave-tc2s"
server_key: "696a663a795f790c60f38437378ea699"
server_key_iv: "286ec1ba2a83b538622b747275260e04"
server_key_version: "encryptionkey@0"
I20260808 11:38:02.132128 32087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance:
uuid: "09a72b38ec7245b382dae73df7f254fb"
format_stamp: "Formatted at 2026-08-08 11:38:02 on dist-test-slave-tc2s"
server_key: "696a663a795f790c60f38437378ea699"
server_key_iv: "286ec1ba2a83b538622b747275260e04"
server_key_version: "encryptionkey@0"
I20260808 11:38:02.136740 32087 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260808 11:38:02.140142 32105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:02.141356 32087 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 11:38:02.141506 32087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-2/wal
uuid: "09a72b38ec7245b382dae73df7f254fb"
format_stamp: "Formatted at 2026-08-08 11:38:02 on dist-test-slave-tc2s"
server_key: "696a663a795f790c60f38437378ea699"
server_key_iv: "286ec1ba2a83b538622b747275260e04"
server_key_version: "encryptionkey@0"
I20260808 11:38:02.141702 32087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:02.154495 32087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:02.158056 32087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:02.158288 32087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:02.159164 32087 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:38:02.160480 32087 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:38:02.160569 32087 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:02.160668 32087 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:38:02.160734 32087 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:02.191372 32087 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.67:36943
I20260808 11:38:02.191411 32283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.67:36943 every 8 connection(s)
I20260808 11:38:02.192196 32087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
I20260808 11:38:02.200017 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 32087
I20260808 11:38:02.200220 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance
I20260808 11:38:02.200779 24881 external_mini_cluster.cc:1428] Setting key 43404c10537553264ad9ae1d1da48cb3
Aug 08 11:38:02 dist-test-slave-tc2s krb5kdc[31508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189082, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:38:02.210856 31869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:02.194565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:52465 (local address 127.24.76.126:34577)
0808 11:38:02.194732 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:02.194737 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:02.195899 (+ 1162us) server_negotiation.cc:407] Connection header received
0808 11:38:02.197651 (+ 1752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:02.197656 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:02.197725 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:02.197810 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:02.199229 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:02.199988 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:02.201986 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:02.202471 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:02.202567 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:02.205998 (+ 3431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:02.206003 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:02.206006 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:02.206040 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:02.208352 (+ 2312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:02.208979 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:02.208982 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:02.208984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:02.209095 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:02.209407 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:02.209410 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:02.209411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:02.209658 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:02.209780 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:02.210413 (+ 633us) server_negotiation.cc:299] Negotiation successful
0808 11:38:02.210577 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 11:38:02.211726 32286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:02.195067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:34577 (local address 127.24.76.67:52465)
0808 11:38:02.195747 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:02.195783 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:02.197393 (+ 1610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:02.198035 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:02.198043 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:02.198553 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:02.199083 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:02.199101 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:02.200133 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:02.200139 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:02.201810 (+ 1671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:02.201827 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:02.202686 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:02.202693 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:02.202911 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:02.203609 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:02.203647 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:02.205777 (+ 2130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:02.208479 (+ 2702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:02.208485 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:02.208496 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:02.208886 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:02.209169 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:02.209172 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:02.209174 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:02.209333 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:02.209773 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:02.209778 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:02.210107 (+ 329us) client_negotiation.cc:769] Sending connection context
0808 11:38:02.210319 (+ 212us) client_negotiation.cc:240] Negotiation successful
0808 11:38:02.210722 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":271,"threads_started":1}
I20260808 11:38:02.212841 32284 heartbeater.cc:347] Connected to a master server at 127.24.76.126:34577
I20260808 11:38:02.213117 32284 heartbeater.cc:464] Registering TS with master...
I20260808 11:38:02.213783 32284 heartbeater.cc:511] Master 127.24.76.126:34577 requested a full tablet report, sending...
I20260808 11:38:02.215337 31578 ts_manager.cc:194] Registered new tserver with Master: 09a72b38ec7245b382dae73df7f254fb (127.24.76.67:36943)
I20260808 11:38:02.216075 24881 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:38:02.216374 31578 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.67@KRBTEST.COM'} at 127.24.76.67:52465
I20260808 11:38:02.232241 31869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:02.220560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48580 (local address 127.24.76.126:34577)
0808 11:38:02.220700 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:02.220704 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:02.221191 (+ 487us) server_negotiation.cc:407] Connection header received
0808 11:38:02.221339 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:02.221344 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:02.221406 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:02.221482 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:02.222873 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:02.223548 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:02.224846 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:02.225297 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:02.225401 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:02.227647 (+ 2246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:02.227652 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:02.227655 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:02.227683 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:02.230049 (+ 2366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:02.230676 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:02.230682 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:02.230686 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:02.230825 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:02.231170 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:02.231173 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:02.231175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:02.231450 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:02.231626 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:02.231875 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 11:38:02.232038 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 11:38:02.235630 31578 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48580:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:38:02.242165 31578 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:38:02.255601 31866 heartbeater.cc:503] Master 127.24.76.126:34577 was elected leader, sending a full tablet report...
I20260808 11:38:02.263109 32307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:02.253109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59292 (local address 127.24.76.66:41513)
0808 11:38:02.254672 (+ 1563us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:02.254681 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:02.254708 (+ 27us) server_negotiation.cc:407] Connection header received
0808 11:38:02.254805 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:02.254813 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:02.255057 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:02.255810 (+ 753us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:38:02.256035 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:02.257475 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:02.260353 (+ 2878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:02.261695 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:02.261828 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:02.262043 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:02.262651 (+ 608us) server_negotiation.cc:299] Negotiation successful
0808 11:38:02.262818 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1404,"thread_start_us":2146,"threads_started":1}
I20260808 11:38:02.263329 32306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:02.252913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:41513 (local address 127.0.0.1:59292)
0808 11:38:02.253941 (+ 1028us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:02.253994 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:02.254170 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:02.255294 (+ 1124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:02.255301 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:02.255331 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:38:02.255633 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:02.255642 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:02.257629 (+ 1987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:02.257634 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:02.260159 (+ 2525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:02.260177 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:02.262169 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:02.262175 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:02.262387 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:02.262449 (+ 62us) client_negotiation.cc:769] Sending connection context
0808 11:38:02.262589 (+ 140us) client_negotiation.cc:240] Negotiation successful
0808 11:38:02.262907 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"mutex_wait_us":96,"thread_start_us":604,"threads_started":1}
I20260808 11:38:02.265084 32308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:02.253129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:44923 (local address 127.0.0.1:43886)
0808 11:38:02.254934 (+ 1805us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:02.254950 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:02.255145 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:02.255888 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:02.255893 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:02.255917 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:38:02.258933 (+ 3016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:02.258948 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:02.260640 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:02.260646 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:02.262612 (+ 1966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:02.262627 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:02.264361 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:02.264372 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:02.264542 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:02.264575 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 11:38:02.264662 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 11:38:02.264776 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1691,"thread_start_us":1553,"threads_started":1}
I20260808 11:38:02.265764 32309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:02.253768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43886 (local address 127.24.76.65:44923)
0808 11:38:02.254511 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:02.254520 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:02.255040 (+ 520us) server_negotiation.cc:407] Connection header received
0808 11:38:02.255345 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:02.255354 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:02.255634 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:02.255793 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:38:02.259159 (+ 3366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:02.260513 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:02.262769 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:02.264135 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:02.264266 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:02.264482 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:02.264741 (+ 259us) server_negotiation.cc:299] Negotiation successful
0808 11:38:02.264898 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"spinlock_wait_cycles":28928,"thread_start_us":558,"threads_started":1}
I20260808 11:38:02.267534 32312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:02.256066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60822 (local address 127.24.76.67:36943)
0808 11:38:02.258729 (+ 2663us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:02.258737 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:02.259155 (+ 418us) server_negotiation.cc:407] Connection header received
0808 11:38:02.259297 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:02.259304 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:02.259515 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:02.259684 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:38:02.260338 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:02.262115 (+ 1777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:02.264577 (+ 2462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:02.265923 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:02.266691 (+ 768us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:02.266899 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:02.267086 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 11:38:02.267239 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2509,"thread_start_us":2347,"threads_started":1}
I20260808 11:38:02.267529 32311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:02.255047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:36943 (local address 127.0.0.1:60822)
0808 11:38:02.259018 (+ 3971us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:02.259035 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:02.259155 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:02.259701 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:02.259706 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:02.259731 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:38:02.260132 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:02.260144 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:02.262251 (+ 2107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:02.262256 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:02.264395 (+ 2139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:02.264410 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:02.266757 (+ 2347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:02.266763 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:02.266923 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:02.266953 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 11:38:02.267032 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 11:38:02.267149 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3845,"mutex_wait_us":21,"thread_start_us":3685,"threads_started":1}
I20260808 11:38:02.267290 31971 tablet_service.cc:1467] Processing CreateTablet for tablet 9c87870967124e27bc72eb318ac0f15b (DEFAULT_TABLE table=test-table [id=acc534d68f4e4436bc6c3eb3de030443]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:38:02.268842 31971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c87870967124e27bc72eb318ac0f15b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:02.269101 31765 tablet_service.cc:1467] Processing CreateTablet for tablet 9c87870967124e27bc72eb318ac0f15b (DEFAULT_TABLE table=test-table [id=acc534d68f4e4436bc6c3eb3de030443]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:38:02.270524 31765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c87870967124e27bc72eb318ac0f15b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:02.270659 32185 tablet_service.cc:1467] Processing CreateTablet for tablet 9c87870967124e27bc72eb318ac0f15b (DEFAULT_TABLE table=test-table [id=acc534d68f4e4436bc6c3eb3de030443]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:38:02.271733 32185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c87870967124e27bc72eb318ac0f15b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:02.282119 32314 tablet_bootstrap.cc:492] T 9c87870967124e27bc72eb318ac0f15b P 09a72b38ec7245b382dae73df7f254fb: Bootstrap starting.
I20260808 11:38:02.284327 32316 tablet_bootstrap.cc:492] T 9c87870967124e27bc72eb318ac0f15b P 439585dddb4040f9809bcdfd0bbaa142: Bootstrap starting.
I20260808 11:38:02.285844 32315 tablet_bootstrap.cc:492] T 9c87870967124e27bc72eb318ac0f15b P e2ca1793304a406b99da5dff38ebf56b: Bootstrap starting.
I20260808 11:38:02.286374 32314 tablet_bootstrap.cc:654] T 9c87870967124e27bc72eb318ac0f15b P 09a72b38ec7245b382dae73df7f254fb: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:02.287644 32316 tablet_bootstrap.cc:654] T 9c87870967124e27bc72eb318ac0f15b P 439585dddb4040f9809bcdfd0bbaa142: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:02.287704 32314 log.cc:824] T 9c87870967124e27bc72eb318ac0f15b P 09a72b38ec7245b382dae73df7f254fb: Log is configured to *not* fsync() on all Append() calls
I20260808 11:38:02.288707 32315 tablet_bootstrap.cc:654] T 9c87870967124e27bc72eb318ac0f15b P e2ca1793304a406b99da5dff38ebf56b: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:02.288769 32316 log.cc:824] T 9c87870967124e27bc72eb318ac0f15b P 439585dddb4040f9809bcdfd0bbaa142: Log is configured to *not* fsync() on all Append() calls
I20260808 11:38:02.289847 32315 log.cc:824] T 9c87870967124e27bc72eb318ac0f15b P e2ca1793304a406b99da5dff38ebf56b: Log is configured to *not* fsync() on all Append() calls
I20260808 11:38:02.290766 32314 tablet_bootstrap.cc:492] T 9c87870967124e27bc72eb318ac0f15b P 09a72b38ec7245b382dae73df7f254fb: No bootstrap required, opened a new log
I20260808 11:38:02.291046 32314 ts_tablet_manager.cc:1397] T 9c87870967124e27bc72eb318ac0f15b P 09a72b38ec7245b382dae73df7f254fb: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260808 11:38:02.292279 32316 tablet_bootstrap.cc:492] T 9c87870967124e27bc72eb318ac0f15b P 439585dddb4040f9809bcdfd0bbaa142: No bootstrap required, opened a new log
I20260808 11:38:02.292424 32315 tablet_bootstrap.cc:492] T 9c87870967124e27bc72eb318ac0f15b P e2ca1793304a406b99da5dff38ebf56b: No bootstrap required, opened a new log
I20260808 11:38:02.292559 32316 ts_tablet_manager.cc:1397] T 9c87870967124e27bc72eb318ac0f15b P 439585dddb4040f9809bcdfd0bbaa142: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20260808 11:38:02.292665 32315 ts_tablet_manager.cc:1397] T 9c87870967124e27bc72eb318ac0f15b P e2ca1793304a406b99da5dff38ebf56b: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 11:38:02.299212 32315 raft_consensus.cc:348] T 9c87870967124e27bc72eb318ac0f15b P e2ca1793304a406b99da5dff38ebf56b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2ca1793304a406b99da5dff38ebf56b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 41513 } } peers { permanent_uuid: "439585dddb4040f9809bcdfd0bbaa142" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44923 } } peers { permanent_uuid: "09a72b38ec7245b382dae73df7f254fb" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 36943 } }
I20260808 11:38:02.299554 32315 raft_consensus.cc:374] T 9c87870967124e27bc72eb318ac0f15b P e2ca1793304a406b99da5dff38ebf56b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:02.299654 32315 raft_consensus.cc:729] T 9c87870967124e27bc72eb318ac0f15b P e2ca1793304a406b99da5dff38ebf56b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2ca1793304a406b99da5dff38ebf56b, State: Initialized, Role: FOLLOWER
I20260808 11:38:02.300187 32315 consensus_queue.cc:260] T 9c87870967124e27bc72eb318ac0f15b P e2ca1793304a406b99da5dff38ebf56b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2ca1793304a406b99da5dff38ebf56b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 41513 } } peers { permanent_uuid: "439585dddb4040f9809bcdfd0bbaa142" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44923 } } peers { permanent_uuid: "09a72b38ec7245b382dae73df7f254fb" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 36943 } }
I20260808 11:38:02.301190 32082 heartbeater.cc:503] Master 127.24.76.126:34577 was elected leader, sending a full tablet report...
I20260808 11:38:02.301077 32314 raft_consensus.cc:348] T 9c87870967124e27bc72eb318ac0f15b P 09a72b38ec7245b382dae73df7f254fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2ca1793304a406b99da5dff38ebf56b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 41513 } } peers { permanent_uuid: "439585dddb4040f9809bcdfd0bbaa142" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44923 } } peers { permanent_uuid: "09a72b38ec7245b382dae73df7f254fb" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 36943 } }
I20260808 11:38:02.301479 32314 raft_consensus.cc:374] T 9c87870967124e27bc72eb318ac0f15b P 09a72b38ec7245b382dae73df7f254fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:02.301582 32314 raft_consensus.cc:729] T 9c87870967124e27bc72eb318ac0f15b P 09a72b38ec7245b382dae73df7f254fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09a72b38ec7245b382dae73df7f254fb, State: Initialized, Role: FOLLOWER
I20260808 11:38:02.302255 32314 consensus_queue.cc:260] T 9c87870967124e27bc72eb318ac0f15b P 09a72b38ec7245b382dae73df7f254fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2ca1793304a406b99da5dff38ebf56b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 41513 } } peers { permanent_uuid: "439585dddb4040f9809bcdfd0bbaa142" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44923 } } peers { permanent_uuid: "09a72b38ec7245b382dae73df7f254fb" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 36943 } }
I20260808 11:38:02.302414 32316 raft_consensus.cc:348] T 9c87870967124e27bc72eb318ac0f15b P 439585dddb4040f9809bcdfd0bbaa142 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2ca1793304a406b99da5dff38ebf56b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 41513 } } peers { permanent_uuid: "439585dddb4040f9809bcdfd0bbaa142" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44923 } } peers { permanent_uuid: "09a72b38ec7245b382dae73df7f254fb" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 36943 } }
I20260808 11:38:02.302875 32316 raft_consensus.cc:374] T 9c87870967124e27bc72eb318ac0f15b P 439585dddb4040f9809bcdfd0bbaa142 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:02.303013 32316 raft_consensus.cc:729] T 9c87870967124e27bc72eb318ac0f15b P 439585dddb4040f9809bcdfd0bbaa142 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 439585dddb4040f9809bcdfd0bbaa142, State: Initialized, Role: FOLLOWER
I20260808 11:38:02.303450 32315 ts_tablet_manager.cc:1428] T 9c87870967124e27bc72eb318ac0f15b P e2ca1793304a406b99da5dff38ebf56b: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260808 11:38:02.303774 32316 consensus_queue.cc:260] T 9c87870967124e27bc72eb318ac0f15b P 439585dddb4040f9809bcdfd0bbaa142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2ca1793304a406b99da5dff38ebf56b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 41513 } } peers { permanent_uuid: "439585dddb4040f9809bcdfd0bbaa142" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44923 } } peers { permanent_uuid: "09a72b38ec7245b382dae73df7f254fb" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 36943 } }
I20260808 11:38:02.305449 32316 ts_tablet_manager.cc:1428] T 9c87870967124e27bc72eb318ac0f15b P 439585dddb4040f9809bcdfd0bbaa142: Time spent starting tablet: real 0.013s user 0.009s sys 0.003s
I20260808 11:38:02.306330 32284 heartbeater.cc:503] Master 127.24.76.126:34577 was elected leader, sending a full tablet report...
I20260808 11:38:02.308104 32314 ts_tablet_manager.cc:1428] T 9c87870967124e27bc72eb318ac0f15b P 09a72b38ec7245b382dae73df7f254fb: Time spent starting tablet: real 0.017s user 0.008s sys 0.008s
W20260808 11:38:02.443607 32285 tablet.cc:2369] T 9c87870967124e27bc72eb318ac0f15b P 09a72b38ec7245b382dae73df7f254fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:38:02.471951 32083 tablet.cc:2369] T 9c87870967124e27bc72eb318ac0f15b P e2ca1793304a406b99da5dff38ebf56b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:38:02.472813 32320 raft_consensus.cc:482] T 9c87870967124e27bc72eb318ac0f15b P e2ca1793304a406b99da5dff38ebf56b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:38:02.473083 32320 raft_consensus.cc:504] T 9c87870967124e27bc72eb318ac0f15b P e2ca1793304a406b99da5dff38ebf56b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2ca1793304a406b99da5dff38ebf56b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 41513 } } peers { permanent_uuid: "439585dddb4040f9809bcdfd0bbaa142" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44923 } } peers { permanent_uuid: "09a72b38ec7245b382dae73df7f254fb" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 36943 } }
I20260808 11:38:02.474735 32320 leader_election.cc:288] T 9c87870967124e27bc72eb318ac0f15b P e2ca1793304a406b99da5dff38ebf56b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 439585dddb4040f9809bcdfd0bbaa142 (127.24.76.65:44923), 09a72b38ec7245b382dae73df7f254fb (127.24.76.67:36943)
W20260808 11:38:02.477406 31867 tablet.cc:2369] T 9c87870967124e27bc72eb318ac0f15b P 439585dddb4040f9809bcdfd0bbaa142: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:38:02.481845 32309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:02.475102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:58419 (local address 127.24.76.65:44923)
0808 11:38:02.475281 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:02.475286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:02.475846 (+ 560us) server_negotiation.cc:407] Connection header received
0808 11:38:02.476084 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:02.476090 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:02.476175 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:02.476279 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:38:02.476840 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:02.477929 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:02.479964 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:02.480890 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:02.481011 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:02.481063 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:02.481476 (+ 413us) server_negotiation.cc:299] Negotiation successful
0808 11:38:02.481603 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 11:38:02.481961 32325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:02.475089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:44923 (local address 127.24.76.66:58419)
0808 11:38:02.475755 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:02.475776 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:02.475916 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:02.476322 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:02.476327 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:02.476355 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:38:02.476692 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:02.476701 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:02.478066 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:02.478071 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:02.479809 (+ 1738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:02.479822 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:02.481094 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:02.481100 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:02.481268 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:02.481348 (+ 80us) client_negotiation.cc:769] Sending connection context
0808 11:38:02.481427 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 11:38:02.481533 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"mutex_wait_us":69,"thread_start_us":361,"threads_started":1}
I20260808 11:38:02.482019 32312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:02.476002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:58339 (local address 127.24.76.67:36943)
0808 11:38:02.476224 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:02.476230 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:02.476628 (+ 398us) server_negotiation.cc:407] Connection header received
0808 11:38:02.476801 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:02.476805 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:02.476863 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:02.476952 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:38:02.477480 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:02.478382 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:02.480179 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:02.481063 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:02.481208 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:02.481277 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:02.481649 (+ 372us) server_negotiation.cc:299] Negotiation successful
0808 11:38:02.481772 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 11:38:02.482154 32327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:02.475880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:36943 (local address 127.24.76.66:58339)
0808 11:38:02.476529 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:02.476548 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:02.476679 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:02.476976 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:02.476980 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:02.477003 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:38:02.477331 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:02.477340 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:02.478518 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:02.478522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:02.480037 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:02.480051 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:02.481266 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:02.481270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:02.481484 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:02.481516 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 11:38:02.481601 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 11:38:02.481721 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"mutex_wait_us":36,"thread_start_us":322,"threads_started":1}
I20260808 11:38:02.483035 31791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c87870967124e27bc72eb318ac0f15b" candidate_uuid: "e2ca1793304a406b99da5dff38ebf56b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "439585dddb4040f9809bcdfd0bbaa142" is_pre_election: true
I20260808 11:38:02.483042 32213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c87870967124e27bc72eb318ac0f15b" candidate_uuid: "e2ca1793304a406b99da5dff38ebf56b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09a72b38ec7245b382dae73df7f254fb" is_pre_election: true
I20260808 11:38:02.483479 32213 raft_consensus.cc:2454] T 9c87870967124e27bc72eb318ac0f15b P 09a72b38ec7245b382dae73df7f254fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2ca1793304a406b99da5dff38ebf56b in term 0.
I20260808 11:38:02.483479 31791 raft_consensus.cc:2454] T 9c87870967124e27bc72eb318ac0f15b P 439585dddb4040f9809bcdfd0bbaa142 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2ca1793304a406b99da5dff38ebf56b in term 0.
I20260808 11:38:02.484066 31894 leader_election.cc:302] T 9c87870967124e27bc72eb318ac0f15b P e2ca1793304a406b99da5dff38ebf56b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 439585dddb4040f9809bcdfd0bbaa142, e2ca1793304a406b99da5dff38ebf56b; no voters:
I20260808 11:38:02.484395 32320 raft_consensus.cc:2781] T 9c87870967124e27bc72eb318ac0f15b P e2ca1793304a406b99da5dff38ebf56b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:38:02.484534 32320 raft_consensus.cc:482] T 9c87870967124e27bc72eb318ac0f15b P e2ca1793304a406b99da5dff38ebf56b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:38:02.484632 32320 raft_consensus.cc:3037] T 9c87870967124e27bc72eb318ac0f15b P e2ca1793304a406b99da5dff38ebf56b [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:02.487118 32320 raft_consensus.cc:504] T 9c87870967124e27bc72eb318ac0f15b P e2ca1793304a406b99da5dff38ebf56b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2ca1793304a406b99da5dff38ebf56b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 41513 } } peers { permanent_uuid: "439585dddb4040f9809bcdfd0bbaa142" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44923 } } peers { permanent_uuid: "09a72b38ec7245b382dae73df7f254fb" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 36943 } }
I20260808 11:38:02.487649 32320 leader_election.cc:288] T 9c87870967124e27bc72eb318ac0f15b P e2ca1793304a406b99da5dff38ebf56b [CANDIDATE]: Term 1 election: Requested vote from peers 439585dddb4040f9809bcdfd0bbaa142 (127.24.76.65:44923), 09a72b38ec7245b382dae73df7f254fb (127.24.76.67:36943)
I20260808 11:38:02.488078 31791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c87870967124e27bc72eb318ac0f15b" candidate_uuid: "e2ca1793304a406b99da5dff38ebf56b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "439585dddb4040f9809bcdfd0bbaa142"
I20260808 11:38:02.488154 32213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c87870967124e27bc72eb318ac0f15b" candidate_uuid: "e2ca1793304a406b99da5dff38ebf56b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09a72b38ec7245b382dae73df7f254fb"
I20260808 11:38:02.488255 31791 raft_consensus.cc:3037] T 9c87870967124e27bc72eb318ac0f15b P 439585dddb4040f9809bcdfd0bbaa142 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:02.488327 32213 raft_consensus.cc:3037] T 9c87870967124e27bc72eb318ac0f15b P 09a72b38ec7245b382dae73df7f254fb [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:02.490434 31791 raft_consensus.cc:2454] T 9c87870967124e27bc72eb318ac0f15b P 439585dddb4040f9809bcdfd0bbaa142 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2ca1793304a406b99da5dff38ebf56b in term 1.
I20260808 11:38:02.490504 32213 raft_consensus.cc:2454] T 9c87870967124e27bc72eb318ac0f15b P 09a72b38ec7245b382dae73df7f254fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2ca1793304a406b99da5dff38ebf56b in term 1.
I20260808 11:38:02.490767 31894 leader_election.cc:302] T 9c87870967124e27bc72eb318ac0f15b P e2ca1793304a406b99da5dff38ebf56b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 439585dddb4040f9809bcdfd0bbaa142, e2ca1793304a406b99da5dff38ebf56b; no voters:
I20260808 11:38:02.491006 32320 raft_consensus.cc:2781] T 9c87870967124e27bc72eb318ac0f15b P e2ca1793304a406b99da5dff38ebf56b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:38:02.491387 32320 raft_consensus.cc:686] T 9c87870967124e27bc72eb318ac0f15b P e2ca1793304a406b99da5dff38ebf56b [term 1 LEADER]: Becoming Leader. State: Replica: e2ca1793304a406b99da5dff38ebf56b, State: Running, Role: LEADER
I20260808 11:38:02.491842 32320 consensus_queue.cc:237] T 9c87870967124e27bc72eb318ac0f15b P e2ca1793304a406b99da5dff38ebf56b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2ca1793304a406b99da5dff38ebf56b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 41513 } } peers { permanent_uuid: "439585dddb4040f9809bcdfd0bbaa142" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44923 } } peers { permanent_uuid: "09a72b38ec7245b382dae73df7f254fb" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 36943 } }
I20260808 11:38:02.496421 31578 catalog_manager.cc:5495] T 9c87870967124e27bc72eb318ac0f15b P e2ca1793304a406b99da5dff38ebf56b reported cstate change: term changed from 0 to 1, leader changed from <none> to e2ca1793304a406b99da5dff38ebf56b (127.24.76.66). New cstate: current_term: 1 leader_uuid: "e2ca1793304a406b99da5dff38ebf56b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2ca1793304a406b99da5dff38ebf56b" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 41513 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "439585dddb4040f9809bcdfd0bbaa142" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09a72b38ec7245b382dae73df7f254fb" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 36943 } health_report { overall_health: UNKNOWN } } }
I20260808 11:38:02.555598 32307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:02.550816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59304 (local address 127.24.76.66:41513)
0808 11:38:02.551021 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:02.551026 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:02.551044 (+ 18us) server_negotiation.cc:407] Connection header received
0808 11:38:02.551156 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:02.551159 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:02.551235 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:02.551352 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:38:02.551671 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:02.552347 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:02.553621 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:02.554078 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:02.554168 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:02.554641 (+ 473us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:38:02.555107 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:38:02.555230 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:02.555265 (+ 35us) server_negotiation.cc:299] Negotiation successful
0808 11:38:02.555327 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 11:38:02.563539 32213 raft_consensus.cc:1261] T 9c87870967124e27bc72eb318ac0f15b P 09a72b38ec7245b382dae73df7f254fb [term 1 FOLLOWER]: Refusing update from remote peer e2ca1793304a406b99da5dff38ebf56b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:38:02.563606 31791 raft_consensus.cc:1261] T 9c87870967124e27bc72eb318ac0f15b P 439585dddb4040f9809bcdfd0bbaa142 [term 1 FOLLOWER]: Refusing update from remote peer e2ca1793304a406b99da5dff38ebf56b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:38:02.564422 32329 consensus_queue.cc:1035] T 9c87870967124e27bc72eb318ac0f15b P e2ca1793304a406b99da5dff38ebf56b [LEADER]: Connected to new peer: Peer: permanent_uuid: "09a72b38ec7245b382dae73df7f254fb" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 36943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:38:02.564759 32320 consensus_queue.cc:1035] T 9c87870967124e27bc72eb318ac0f15b P e2ca1793304a406b99da5dff38ebf56b [LEADER]: Connected to new peer: Peer: permanent_uuid: "439585dddb4040f9809bcdfd0bbaa142" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:38:02.588721 31578 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:38:02.589020 31578 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:38:02.592465 31578 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 823a77169eaa400cac437f026fa98059: Sending DeleteTablet for 3 replicas of tablet 9c87870967124e27bc72eb318ac0f15b
I20260808 11:38:02.593503 31971 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c87870967124e27bc72eb318ac0f15b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:38:02 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:59292
I20260808 11:38:02.593950 31765 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c87870967124e27bc72eb318ac0f15b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:38:02 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:43886
I20260808 11:38:02.594523 32185 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c87870967124e27bc72eb318ac0f15b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:38:02 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:60822
I20260808 11:38:02.596359 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 31668
I20260808 11:38:02.603430 32356 tablet_replica.cc:330] T 9c87870967124e27bc72eb318ac0f15b P 09a72b38ec7245b382dae73df7f254fb: stopping tablet replica
I20260808 11:38:02.603574 32353 tablet_replica.cc:330] T 9c87870967124e27bc72eb318ac0f15b P e2ca1793304a406b99da5dff38ebf56b: stopping tablet replica
I20260808 11:38:02.603879 32356 raft_consensus.cc:2229] T 9c87870967124e27bc72eb318ac0f15b P 09a72b38ec7245b382dae73df7f254fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:38:02.603899 32353 raft_consensus.cc:2229] T 9c87870967124e27bc72eb318ac0f15b P e2ca1793304a406b99da5dff38ebf56b [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:38:02.604380 32356 raft_consensus.cc:2258] T 9c87870967124e27bc72eb318ac0f15b P 09a72b38ec7245b382dae73df7f254fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:38:02.604384 32353 raft_consensus.cc:2258] T 9c87870967124e27bc72eb318ac0f15b P e2ca1793304a406b99da5dff38ebf56b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:38:02.606384 32356 ts_tablet_manager.cc:1905] T 9c87870967124e27bc72eb318ac0f15b P 09a72b38ec7245b382dae73df7f254fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:38:02.606379 32353 ts_tablet_manager.cc:1905] T 9c87870967124e27bc72eb318ac0f15b P e2ca1793304a406b99da5dff38ebf56b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:38:02.608227 32355 tablet_replica.cc:330] T 9c87870967124e27bc72eb318ac0f15b P 439585dddb4040f9809bcdfd0bbaa142: stopping tablet replica
I20260808 11:38:02.608611 32355 raft_consensus.cc:2229] T 9c87870967124e27bc72eb318ac0f15b P 439585dddb4040f9809bcdfd0bbaa142 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:38:02.609061 32355 raft_consensus.cc:2258] T 9c87870967124e27bc72eb318ac0f15b P 439585dddb4040f9809bcdfd0bbaa142 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:38:02.611140 32355 ts_tablet_manager.cc:1905] T 9c87870967124e27bc72eb318ac0f15b P 439585dddb4040f9809bcdfd0bbaa142: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:38:02.613525 32356 ts_tablet_manager.cc:1918] T 9c87870967124e27bc72eb318ac0f15b P 09a72b38ec7245b382dae73df7f254fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:38:02.613732 32356 log.cc:1196] T 9c87870967124e27bc72eb318ac0f15b P 09a72b38ec7245b382dae73df7f254fb: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-2/wal/wals/9c87870967124e27bc72eb318ac0f15b
I20260808 11:38:02.614562 32356 ts_tablet_manager.cc:1939] T 9c87870967124e27bc72eb318ac0f15b P 09a72b38ec7245b382dae73df7f254fb: Deleting consensus metadata
I20260808 11:38:02.616725 31554 catalog_manager.cc:4841] TS 09a72b38ec7245b382dae73df7f254fb (127.24.76.67:36943): tablet 9c87870967124e27bc72eb318ac0f15b (table test-table [id=acc534d68f4e4436bc6c3eb3de030443]) successfully deleted
I20260808 11:38:02.617252 32355 ts_tablet_manager.cc:1918] T 9c87870967124e27bc72eb318ac0f15b P 439585dddb4040f9809bcdfd0bbaa142: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:38:02.617414 32355 log.cc:1196] T 9c87870967124e27bc72eb318ac0f15b P 439585dddb4040f9809bcdfd0bbaa142: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-0/wal/wals/9c87870967124e27bc72eb318ac0f15b
I20260808 11:38:02.617960 32355 ts_tablet_manager.cc:1939] T 9c87870967124e27bc72eb318ac0f15b P 439585dddb4040f9809bcdfd0bbaa142: Deleting consensus metadata
I20260808 11:38:02.619376 32353 ts_tablet_manager.cc:1918] T 9c87870967124e27bc72eb318ac0f15b P e2ca1793304a406b99da5dff38ebf56b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:38:02.619390 31554 catalog_manager.cc:4841] TS 439585dddb4040f9809bcdfd0bbaa142 (127.24.76.65:44923): tablet 9c87870967124e27bc72eb318ac0f15b (table test-table [id=acc534d68f4e4436bc6c3eb3de030443]) successfully deleted
I20260808 11:38:02.619552 32353 log.cc:1196] T 9c87870967124e27bc72eb318ac0f15b P e2ca1793304a406b99da5dff38ebf56b: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786189042529715-24881-0/minicluster-data/ts-1/wal/wals/9c87870967124e27bc72eb318ac0f15b
I20260808 11:38:02.619995 32353 ts_tablet_manager.cc:1939] T 9c87870967124e27bc72eb318ac0f15b P e2ca1793304a406b99da5dff38ebf56b: Deleting consensus metadata
I20260808 11:38:02.621232 31558 catalog_manager.cc:4841] TS e2ca1793304a406b99da5dff38ebf56b (127.24.76.66:41513): tablet 9c87870967124e27bc72eb318ac0f15b (table test-table [id=acc534d68f4e4436bc6c3eb3de030443]) successfully deleted
Aug 08 11:38:02 dist-test-slave-tc2s krb5kdc[31508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189080, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.65@KRBTEST.COM
I20260808 11:38:02.628535 32309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:02.597747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43888 (local address 127.24.76.65:44923)
0808 11:38:02.597944 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:02.597952 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:02.612618 (+ 14666us) server_negotiation.cc:407] Connection header received
0808 11:38:02.612792 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:02.612797 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:02.612879 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:02.612978 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:02.614837 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:02.616050 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:02.617831 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:02.618507 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:02.618626 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:02.623236 (+ 4610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:02.623247 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:02.623259 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:02.623290 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:02.625851 (+ 2561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:02.626506 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:02.626513 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:02.626515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:02.626714 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:02.627073 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:02.627077 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:02.627079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:02.627420 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:02.627731 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:02.627984 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 11:38:02.628195 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 11:38:02.629101 31859 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:02.728818 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 31668
I20260808 11:38:02.752689 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 31875
Aug 08 11:38:02 dist-test-slave-tc2s krb5kdc[31508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189080, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.66@KRBTEST.COM
I20260808 11:38:02.769366 32307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:02.753413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37316 (local address 127.24.76.66:41513)
0808 11:38:02.753581 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:02.753585 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:02.753640 (+ 55us) server_negotiation.cc:407] Connection header received
0808 11:38:02.753860 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:02.753864 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:02.753976 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:02.754056 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:02.755645 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:02.756480 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:02.758313 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:02.758738 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:02.758827 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:02.763753 (+ 4926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:02.763767 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:02.763785 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:02.763828 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:02.766864 (+ 3036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:02.767597 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:02.767604 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:02.767606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:02.767728 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:02.768085 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:02.768091 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:02.768093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:02.768428 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:02.768695 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:02.768935 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 11:38:02.769090 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 11:38:02.770056 32073 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:02.879987 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 31875
I20260808 11:38:02.903090 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 32087
Aug 08 11:38:02 dist-test-slave-tc2s krb5kdc[31508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189080, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.67@KRBTEST.COM
I20260808 11:38:02.917186 32312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:02.903796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57128 (local address 127.24.76.67:36943)
0808 11:38:02.903948 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:02.903952 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:02.903998 (+ 46us) server_negotiation.cc:407] Connection header received
0808 11:38:02.904146 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:02.904149 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:02.904225 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:02.904307 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:02.905495 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:02.906251 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:02.907427 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:02.907788 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:02.907874 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:02.912063 (+ 4189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:02.912078 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:02.912094 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:02.912126 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:02.914854 (+ 2728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:02.915406 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:02.915414 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:02.915417 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:02.915540 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:02.915845 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:02.915849 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:02.915850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:02.916232 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:02.916459 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:02.916694 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 11:38:02.916859 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 11:38:02.917791 32278 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:03.003494 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 32087
I20260808 11:38:03.022826 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 31531
I20260808 11:38:03.023897 31622 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:03.131373 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 31531
2026-08-08T11:38:03Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4280 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T11:38:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:38:03Z Disabled control of system clock
I20260808 11:38:03.207903 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:41113
--webserver_interface=127.24.76.126
--webserver_port=0
--builtin_ntp_servers=127.24.76.84:35231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:41113
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 11:38:03.526607 32367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:03.526899 32367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:03.526993 32367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:03.533991 32367 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:38:03.534087 32367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:03.534132 32367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:03.534188 32367 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:38:03.534232 32367 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:38:03.543201 32367 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:35231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:41113
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:41113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:03.544358 32367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:03.546123 32367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:03.557639 32373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:03.557565 32378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:03.557613 32374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:03.558632 32367 server_base.cc:1034] running on GCE node
I20260808 11:38:03.559409 32367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:03.561087 32367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:03.562282 32367 hybrid_clock.cc:648] HybridClock initialized: now 1786189083562232 us; error 46 us; skew 500 ppm
I20260808 11:38:03.562640 32367 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:38:03.564867 32367 webserver.cc:460] Webserver started at http://127.24.76.126:37999/ using document root <none> and password file <none>
I20260808 11:38:03.565536 32367 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:03.565634 32367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:03.565987 32367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:03.568812 32367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "7256926d80b240b18618eea079af9ed4"
format_stamp: "Formatted at 2026-08-08 11:38:03 on dist-test-slave-tc2s"
server_key: "947b16d42d6a40d3e8d593f15ff63f42"
server_key_iv: "0690d1f4e45af747beff24715f449ff7"
server_key_version: "encryptionkey@0"
I20260808 11:38:03.569617 32367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "7256926d80b240b18618eea079af9ed4"
format_stamp: "Formatted at 2026-08-08 11:38:03 on dist-test-slave-tc2s"
server_key: "947b16d42d6a40d3e8d593f15ff63f42"
server_key_iv: "0690d1f4e45af747beff24715f449ff7"
server_key_version: "encryptionkey@0"
I20260808 11:38:03.574556 32367 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260808 11:38:03.577883 32387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:03.579170 32367 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260808 11:38:03.579329 32367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "7256926d80b240b18618eea079af9ed4"
format_stamp: "Formatted at 2026-08-08 11:38:03 on dist-test-slave-tc2s"
server_key: "947b16d42d6a40d3e8d593f15ff63f42"
server_key_iv: "0690d1f4e45af747beff24715f449ff7"
server_key_version: "encryptionkey@0"
I20260808 11:38:03.579519 32367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:03.584962 32367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:03.585671 32367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:03.586014 32367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:03.602816 32367 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:41113
I20260808 11:38:03.602859 32465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:41113 every 8 connection(s)
I20260808 11:38:03.604038 32367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:38:03.604538 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 32367
I20260808 11:38:03.604718 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:38:03.605481 24881 external_mini_cluster.cc:1428] Setting key be513cfe07406af9c2ffb9db75dc1568
I20260808 11:38:03.608199 32466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:03.614799 32469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:03.607351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36198 (local address 127.24.76.126:41113)
0808 11:38:03.608458 (+ 1107us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:03.608474 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:03.608523 (+ 49us) server_negotiation.cc:407] Connection header received
0808 11:38:03.610841 (+ 2318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:03.610866 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:03.611383 (+ 517us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:03.611793 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:03.612793 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:03.612800 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:03.612816 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:38:03.612831 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:03.612932 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:38:03.612960 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:03.613182 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:03.613356 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 11:38:03.613626 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":356,"threads_started":1}
I20260808 11:38:03.620826 32466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7256926d80b240b18618eea079af9ed4: Bootstrap starting.
I20260808 11:38:03.623626 32466 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7256926d80b240b18618eea079af9ed4: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:03.624554 32466 log.cc:824] T 00000000000000000000000000000000 P 7256926d80b240b18618eea079af9ed4: Log is configured to *not* fsync() on all Append() calls
I20260808 11:38:03.627007 32466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7256926d80b240b18618eea079af9ed4: No bootstrap required, opened a new log
I20260808 11:38:03.633797 32466 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7256926d80b240b18618eea079af9ed4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7256926d80b240b18618eea079af9ed4" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41113 } }
I20260808 11:38:03.634150 32466 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7256926d80b240b18618eea079af9ed4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:03.634233 32466 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7256926d80b240b18618eea079af9ed4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7256926d80b240b18618eea079af9ed4, State: Initialized, Role: FOLLOWER
I20260808 11:38:03.634719 32466 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7256926d80b240b18618eea079af9ed4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7256926d80b240b18618eea079af9ed4" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41113 } }
I20260808 11:38:03.634931 32466 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7256926d80b240b18618eea079af9ed4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:38:03.635051 32466 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7256926d80b240b18618eea079af9ed4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:38:03.635206 32466 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7256926d80b240b18618eea079af9ed4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:03.637133 32466 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7256926d80b240b18618eea079af9ed4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7256926d80b240b18618eea079af9ed4" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41113 } }
I20260808 11:38:03.637584 32466 leader_election.cc:302] T 00000000000000000000000000000000 P 7256926d80b240b18618eea079af9ed4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7256926d80b240b18618eea079af9ed4; no voters:
I20260808 11:38:03.638164 32466 leader_election.cc:288] T 00000000000000000000000000000000 P 7256926d80b240b18618eea079af9ed4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:38:03.638269 32471 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7256926d80b240b18618eea079af9ed4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:38:03.638926 32471 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7256926d80b240b18618eea079af9ed4 [term 1 LEADER]: Becoming Leader. State: Replica: 7256926d80b240b18618eea079af9ed4, State: Running, Role: LEADER
I20260808 11:38:03.639436 32471 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7256926d80b240b18618eea079af9ed4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7256926d80b240b18618eea079af9ed4" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41113 } }
I20260808 11:38:03.640002 32466 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7256926d80b240b18618eea079af9ed4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:38:03.642176 32472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7256926d80b240b18618eea079af9ed4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7256926d80b240b18618eea079af9ed4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7256926d80b240b18618eea079af9ed4" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41113 } } }
I20260808 11:38:03.642204 32473 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7256926d80b240b18618eea079af9ed4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7256926d80b240b18618eea079af9ed4. Latest consensus state: current_term: 1 leader_uuid: "7256926d80b240b18618eea079af9ed4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7256926d80b240b18618eea079af9ed4" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41113 } } }
I20260808 11:38:03.642619 32472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7256926d80b240b18618eea079af9ed4 [sys.catalog]: This master's current role is: LEADER
I20260808 11:38:03.642616 32473 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7256926d80b240b18618eea079af9ed4 [sys.catalog]: This master's current role is: LEADER
I20260808 11:38:03.644819 32479 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:38:03.649447 32479 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:38:03.658010 32479 catalog_manager.cc:1347] Generated new cluster ID: c10e6a08879f4b6f8ab225e0a8c7f9a3
I20260808 11:38:03.658125 32479 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:38:03.679119 32479 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:38:03.679477 32479 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:38:03.688196 32479 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7256926d80b240b18618eea079af9ed4: Generated new TSK 0
I20260808 11:38:03.688887 32479 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:38:03.698859 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.65:0
--local_ip_for_outbound_sockets=127.24.76.65
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:41113
--builtin_ntp_servers=127.24.76.84:35231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 11:38:04.023634 32494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:04.023878 32494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:04.024004 32494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:04.030954 32494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:04.031046 32494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:04.031193 32494 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.65
I20260808 11:38:04.039924 32494 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:35231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:41113
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.65
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:04.041201 32494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:04.043002 32494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:04.055382 32501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:04.055430 32503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:04.055881 32505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:04.056146 32494 server_base.cc:1034] running on GCE node
I20260808 11:38:04.056728 32494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:04.057597 32494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:04.058787 32494 hybrid_clock.cc:648] HybridClock initialized: now 1786189084058752 us; error 25 us; skew 500 ppm
I20260808 11:38:04.059077 32494 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:38:04.064834 32494 webserver.cc:460] Webserver started at http://127.24.76.65:44771/ using document root <none> and password file <none>
I20260808 11:38:04.065475 32494 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:04.065577 32494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:04.065907 32494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:04.068605 32494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-0/data/instance:
uuid: "15e071eb5c6e413d9bc3d993886ed27d"
format_stamp: "Formatted at 2026-08-08 11:38:04 on dist-test-slave-tc2s"
server_key: "a10c8c13de837583ced4274d0fd0d4df"
server_key_iv: "54c4413e9010532420e5b3a3ce3b5c9e"
server_key_version: "encryptionkey@0"
I20260808 11:38:04.069355 32494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance:
uuid: "15e071eb5c6e413d9bc3d993886ed27d"
format_stamp: "Formatted at 2026-08-08 11:38:04 on dist-test-slave-tc2s"
server_key: "a10c8c13de837583ced4274d0fd0d4df"
server_key_iv: "54c4413e9010532420e5b3a3ce3b5c9e"
server_key_version: "encryptionkey@0"
I20260808 11:38:04.074215 32494 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260808 11:38:04.077405 32514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:04.078619 32494 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 11:38:04.078756 32494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-0/wal
uuid: "15e071eb5c6e413d9bc3d993886ed27d"
format_stamp: "Formatted at 2026-08-08 11:38:04 on dist-test-slave-tc2s"
server_key: "a10c8c13de837583ced4274d0fd0d4df"
server_key_iv: "54c4413e9010532420e5b3a3ce3b5c9e"
server_key_version: "encryptionkey@0"
I20260808 11:38:04.078944 32494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:04.084241 32494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:04.084967 32494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:04.085206 32494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:04.086061 32494 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:38:04.087368 32494 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:38:04.087469 32494 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:04.087568 32494 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:38:04.087636 32494 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:04.116750 32494 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.65:40983
I20260808 11:38:04.116796 32705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.65:40983 every 8 connection(s)
I20260808 11:38:04.118063 32494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
I20260808 11:38:04.121057 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 32494
I20260808 11:38:04.121220 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance
I20260808 11:38:04.121841 24881 external_mini_cluster.cc:1428] Setting key 8b26a639f4a95fa9e4fe0d6725fafef5
I20260808 11:38:04.124693 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.66:0
--local_ip_for_outbound_sockets=127.24.76.66
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:41113
--builtin_ntp_servers=127.24.76.84:35231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 11:38:04.126442 32710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:04.120498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:42981 (local address 127.24.76.126:41113)
0808 11:38:04.120988 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:04.120994 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:04.122301 (+ 1307us) server_negotiation.cc:407] Connection header received
0808 11:38:04.124305 (+ 2004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:04.124310 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:04.124372 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:04.124489 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:04.125673 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:04.125679 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:04.125685 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:38:04.125704 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:04.125769 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:38:04.125785 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:04.125918 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:04.126101 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 11:38:04.126162 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":230,"threads_started":1}
I20260808 11:38:04.127187 32711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:04.121106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:41113 (local address 127.24.76.65:42981)
0808 11:38:04.122141 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:04.122184 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:04.124064 (+ 1880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:04.124657 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:04.124664 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:04.124747 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:04.125382 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:38:04.125384 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:04.125419 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:38:04.125510 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:04.125886 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:04.125916 (+ 30us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:04.125925 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 11:38:04.126142 (+ 217us) client_negotiation.cc:240] Negotiation successful
0808 11:38:04.126321 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":298,"threads_started":1}
I20260808 11:38:04.128522 32706 heartbeater.cc:347] Connected to a master server at 127.24.76.126:41113
I20260808 11:38:04.128780 32706 heartbeater.cc:464] Registering TS with master...
I20260808 11:38:04.129307 32706 heartbeater.cc:511] Master 127.24.76.126:41113 requested a full tablet report, sending...
I20260808 11:38:04.131214 32410 ts_manager.cc:194] Registered new tserver with Master: 15e071eb5c6e413d9bc3d993886ed27d (127.24.76.65:40983)
W20260808 11:38:04.444144 32713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:04.444407 32713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:04.444542 32713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:04.451683 32713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:04.451772 32713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:04.451915 32713 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.66
I20260808 11:38:04.460597 32713 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:35231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:41113
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.66
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:04.461730 32713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:04.463431 32713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:04.475427 32725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:04.475342 32719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:04.475404 32720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:04.476486 32713 server_base.cc:1034] running on GCE node
I20260808 11:38:04.477639 32713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:04.478531 32713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:04.479729 32713 hybrid_clock.cc:648] HybridClock initialized: now 1786189084479676 us; error 48 us; skew 500 ppm
I20260808 11:38:04.480067 32713 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:38:04.482134 32713 webserver.cc:460] Webserver started at http://127.24.76.66:41785/ using document root <none> and password file <none>
I20260808 11:38:04.482792 32713 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:04.482890 32713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:04.483203 32713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:04.485917 32713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-1/data/instance:
uuid: "7c07f52546064127addc99be0ee20f3c"
format_stamp: "Formatted at 2026-08-08 11:38:04 on dist-test-slave-tc2s"
server_key: "91613ef420b2a96d416650dfeac9b15b"
server_key_iv: "427f32c1cf904832437c77eb15c58012"
server_key_version: "encryptionkey@0"
I20260808 11:38:04.486734 32713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance:
uuid: "7c07f52546064127addc99be0ee20f3c"
format_stamp: "Formatted at 2026-08-08 11:38:04 on dist-test-slave-tc2s"
server_key: "91613ef420b2a96d416650dfeac9b15b"
server_key_iv: "427f32c1cf904832437c77eb15c58012"
server_key_version: "encryptionkey@0"
I20260808 11:38:04.491505 32713 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 11:38:04.494678 32734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:04.495838 32713 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 11:38:04.495983 32713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-1/wal
uuid: "7c07f52546064127addc99be0ee20f3c"
format_stamp: "Formatted at 2026-08-08 11:38:04 on dist-test-slave-tc2s"
server_key: "91613ef420b2a96d416650dfeac9b15b"
server_key_iv: "427f32c1cf904832437c77eb15c58012"
server_key_version: "encryptionkey@0"
I20260808 11:38:04.496223 32713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:04.501364 32713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:04.502135 32713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:04.502367 32713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:04.503187 32713 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:38:04.504462 32713 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:38:04.504558 32713 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:04.504662 32713 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:38:04.504733 32713 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:04.538043 32713 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.66:40111
I20260808 11:38:04.538089 447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.66:40111 every 8 connection(s)
I20260808 11:38:04.539318 32713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
I20260808 11:38:04.546693 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 32713
I20260808 11:38:04.546864 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance
I20260808 11:38:04.547323 24881 external_mini_cluster.cc:1428] Setting key bb4b14de0a9883476b4c7af5c0e39b71
I20260808 11:38:04.548050 32710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:04.542141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:44307 (local address 127.24.76.126:41113)
0808 11:38:04.542326 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:04.542331 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:04.543996 (+ 1665us) server_negotiation.cc:407] Connection header received
0808 11:38:04.545760 (+ 1764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:04.545766 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:04.545837 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:04.545970 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:04.547311 (+ 1341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:04.547315 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:04.547319 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:38:04.547335 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:04.547367 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:38:04.547379 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:04.547490 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:04.547746 (+ 256us) server_negotiation.cc:299] Negotiation successful
0808 11:38:04.547827 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 11:38:04.548936 451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:04.542841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:41113 (local address 127.24.76.66:44307)
0808 11:38:04.543832 (+ 991us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:04.543874 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:04.545448 (+ 1574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:04.546197 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:04.546204 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:04.546294 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:04.547021 (+ 727us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:38:04.547023 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:04.547060 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:38:04.547146 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:04.547536 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:04.547545 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:04.547551 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 11:38:04.547756 (+ 205us) client_negotiation.cc:240] Negotiation successful
0808 11:38:04.547991 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":343,"threads_started":1}
I20260808 11:38:04.550323 448 heartbeater.cc:347] Connected to a master server at 127.24.76.126:41113
I20260808 11:38:04.550338 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.67:0
--local_ip_for_outbound_sockets=127.24.76.67
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:41113
--builtin_ntp_servers=127.24.76.84:35231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 11:38:04.550594 448 heartbeater.cc:464] Registering TS with master...
I20260808 11:38:04.551146 448 heartbeater.cc:511] Master 127.24.76.126:41113 requested a full tablet report, sending...
I20260808 11:38:04.552551 32410 ts_manager.cc:194] Registered new tserver with Master: 7c07f52546064127addc99be0ee20f3c (127.24.76.66:40111)
W20260808 11:38:04.876556 452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:04.876812 452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:04.876909 452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:04.884146 452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:04.884240 452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:04.884362 452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.67
I20260808 11:38:04.893143 452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:35231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:41113
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.67
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:04.894408 452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:04.896135 452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:04.908996 463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:04.909005 469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:04.908957 465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:04.909655 452 server_base.cc:1034] running on GCE node
I20260808 11:38:04.910267 452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:04.911149 452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:04.912334 452 hybrid_clock.cc:648] HybridClock initialized: now 1786189084912294 us; error 33 us; skew 500 ppm
I20260808 11:38:04.912637 452 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:38:04.914752 452 webserver.cc:460] Webserver started at http://127.24.76.67:32797/ using document root <none> and password file <none>
I20260808 11:38:04.915376 452 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:04.915479 452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:04.915791 452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:04.918653 452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-2/data/instance:
uuid: "36689af90e274323986170dc9d9ce213"
format_stamp: "Formatted at 2026-08-08 11:38:04 on dist-test-slave-tc2s"
server_key: "da5a21008b76e97cb5ca32474ad5f40f"
server_key_iv: "13f721c301ab7e30f4ccdc486539dc51"
server_key_version: "encryptionkey@0"
I20260808 11:38:04.919435 452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance:
uuid: "36689af90e274323986170dc9d9ce213"
format_stamp: "Formatted at 2026-08-08 11:38:04 on dist-test-slave-tc2s"
server_key: "da5a21008b76e97cb5ca32474ad5f40f"
server_key_iv: "13f721c301ab7e30f4ccdc486539dc51"
server_key_version: "encryptionkey@0"
I20260808 11:38:04.924347 452 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 11:38:04.927598 479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:04.928764 452 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 11:38:04.928902 452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-2/wal
uuid: "36689af90e274323986170dc9d9ce213"
format_stamp: "Formatted at 2026-08-08 11:38:04 on dist-test-slave-tc2s"
server_key: "da5a21008b76e97cb5ca32474ad5f40f"
server_key_iv: "13f721c301ab7e30f4ccdc486539dc51"
server_key_version: "encryptionkey@0"
I20260808 11:38:04.929095 452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:04.934257 452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:04.934973 452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:04.935204 452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:04.936038 452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:38:04.937305 452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:38:04.937399 452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:04.937506 452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:38:04.937574 452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:04.966509 452 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.67:46561
I20260808 11:38:04.966542 652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.67:46561 every 8 connection(s)
I20260808 11:38:04.967782 452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
I20260808 11:38:04.973030 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 452
I20260808 11:38:04.973182 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance
I20260808 11:38:04.973627 24881 external_mini_cluster.cc:1428] Setting key f0700b2aa15cc3569fe0186d60ffde25
I20260808 11:38:04.975765 32710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:04.970229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:44639 (local address 127.24.76.126:41113)
0808 11:38:04.970410 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:04.970414 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:04.971760 (+ 1346us) server_negotiation.cc:407] Connection header received
0808 11:38:04.973446 (+ 1686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:04.973452 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:04.973522 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:04.973606 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:04.974992 (+ 1386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:04.974998 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:04.975003 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:38:04.975023 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:04.975055 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:38:04.975068 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:04.975181 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:04.975410 (+ 229us) server_negotiation.cc:299] Negotiation successful
0808 11:38:04.975479 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 11:38:04.976615 659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:04.970741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:41113 (local address 127.24.76.67:44639)
0808 11:38:04.971604 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:04.971646 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:04.973159 (+ 1513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:04.973839 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:04.973847 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:04.973970 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:04.974677 (+ 707us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:38:04.974680 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:04.974730 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:38:04.974828 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:04.975157 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:04.975169 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:04.975177 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 11:38:04.975482 (+ 305us) client_negotiation.cc:240] Negotiation successful
0808 11:38:04.975718 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":265,"threads_started":1}
I20260808 11:38:04.977777 656 heartbeater.cc:347] Connected to a master server at 127.24.76.126:41113
I20260808 11:38:04.978060 656 heartbeater.cc:464] Registering TS with master...
I20260808 11:38:04.978596 656 heartbeater.cc:511] Master 127.24.76.126:41113 requested a full tablet report, sending...
I20260808 11:38:04.979918 32410 ts_manager.cc:194] Registered new tserver with Master: 36689af90e274323986170dc9d9ce213 (127.24.76.67:46561)
I20260808 11:38:04.988461 24881 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:38:04.994932 32710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:04.992705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36208 (local address 127.24.76.126:41113)
0808 11:38:04.992878 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:04.992884 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:04.993376 (+ 492us) server_negotiation.cc:407] Connection header received
0808 11:38:04.993496 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:04.993500 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:04.993549 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:04.993622 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:04.994457 (+ 835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:04.994461 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:04.994464 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:38:04.994478 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:04.994508 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:38:04.994520 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:04.994610 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:04.994642 (+ 32us) server_negotiation.cc:299] Negotiation successful
0808 11:38:04.994685 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 11:38:04.998543 32410 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36208:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:38:05.005344 32410 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:38:05.036288 675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:05.016199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:40111 (local address 127.0.0.1:49992)
0808 11:38:05.023713 (+ 7514us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:05.023771 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:05.023912 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:05.033121 (+ 9209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:05.033130 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:05.033228 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:05.034251 (+ 1023us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:38:05.034255 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:05.034287 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:38:05.034382 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:05.035653 (+ 1271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:05.035665 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:05.035673 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 11:38:05.035812 (+ 139us) client_negotiation.cc:240] Negotiation successful
0808 11:38:05.035910 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7227,"thread_start_us":7072,"threads_started":1}
I20260808 11:38:05.037302 676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:05.016509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49992 (local address 127.24.76.66:40111)
0808 11:38:05.032306 (+ 15797us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:05.032316 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:05.032352 (+ 36us) server_negotiation.cc:407] Connection header received
0808 11:38:05.032431 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:05.032440 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:05.032698 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:05.033922 (+ 1224us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:05.035402 (+ 1480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:05.035410 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:05.035424 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:38:05.035456 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:05.035512 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:38:05.035533 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:05.036783 (+ 1250us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:05.036859 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 11:38:05.036963 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15603,"thread_start_us":16316,"threads_started":1}
I20260808 11:38:05.037307 679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:05.016885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:46561 (local address 127.0.0.1:41262)
0808 11:38:05.033775 (+ 16890us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:05.034239 (+ 464us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:05.034335 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:05.034930 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:05.034935 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:05.034968 (+ 33us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:05.035576 (+ 608us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:38:05.035580 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:05.035595 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:38:05.035637 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:05.036669 (+ 1032us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:05.036674 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:05.036679 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 11:38:05.036818 (+ 139us) client_negotiation.cc:240] Negotiation successful
0808 11:38:05.036870 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16700,"thread_start_us":16533,"threads_started":1}
I20260808 11:38:05.037950 677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:05.016591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:40983 (local address 127.0.0.1:47344)
0808 11:38:05.028689 (+ 12098us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:05.028713 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:05.028809 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:05.035307 (+ 6498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:05.035312 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:05.035334 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:05.035907 (+ 573us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:38:05.035911 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:05.035925 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:38:05.035976 (+ 51us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:05.037209 (+ 1233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:05.037214 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:05.037218 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 11:38:05.037446 (+ 228us) client_negotiation.cc:240] Negotiation successful
0808 11:38:05.037498 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11950,"thread_start_us":11791,"threads_started":1}
I20260808 11:38:05.037995 682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:05.017387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41262 (local address 127.24.76.67:46561)
0808 11:38:05.033250 (+ 15863us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:05.033259 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:05.034450 (+ 1191us) server_negotiation.cc:407] Connection header received
0808 11:38:05.034543 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:05.034551 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:05.034758 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:05.034904 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:05.036314 (+ 1410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:05.036323 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:05.036335 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:38:05.036350 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:05.036407 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:38:05.036427 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:05.036606 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:05.037529 (+ 923us) server_negotiation.cc:299] Negotiation successful
0808 11:38:05.037621 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15696,"thread_start_us":15436,"threads_started":1}
I20260808 11:38:05.038226 680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:05.017183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47344 (local address 127.24.76.65:40983)
0808 11:38:05.034814 (+ 17631us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:05.034821 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:05.034848 (+ 27us) server_negotiation.cc:407] Connection header received
0808 11:38:05.034913 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:05.034919 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:05.035115 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:05.035712 (+ 597us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:05.036905 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:05.036912 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:05.036923 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:38:05.036948 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:05.037008 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:38:05.037027 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:05.037151 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:05.037815 (+ 664us) server_negotiation.cc:299] Negotiation successful
0808 11:38:05.037931 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17477,"thread_start_us":17358,"threads_started":1}
I20260808 11:38:05.041518 32593 tablet_service.cc:1467] Processing CreateTablet for tablet d9373f55da9c409792ec85c46bc9783e (DEFAULT_TABLE table=test-table [id=7bca775500164b5495b40c7af343928a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:38:05.041422 556 tablet_service.cc:1467] Processing CreateTablet for tablet d9373f55da9c409792ec85c46bc9783e (DEFAULT_TABLE table=test-table [id=7bca775500164b5495b40c7af343928a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:38:05.041424 348 tablet_service.cc:1467] Processing CreateTablet for tablet d9373f55da9c409792ec85c46bc9783e (DEFAULT_TABLE table=test-table [id=7bca775500164b5495b40c7af343928a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:38:05.042714 32593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9373f55da9c409792ec85c46bc9783e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:05.042773 348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9373f55da9c409792ec85c46bc9783e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:05.042773 556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9373f55da9c409792ec85c46bc9783e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:05.054000 683 tablet_bootstrap.cc:492] T d9373f55da9c409792ec85c46bc9783e P 15e071eb5c6e413d9bc3d993886ed27d: Bootstrap starting.
I20260808 11:38:05.056766 684 tablet_bootstrap.cc:492] T d9373f55da9c409792ec85c46bc9783e P 7c07f52546064127addc99be0ee20f3c: Bootstrap starting.
I20260808 11:38:05.058161 683 tablet_bootstrap.cc:654] T d9373f55da9c409792ec85c46bc9783e P 15e071eb5c6e413d9bc3d993886ed27d: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:05.058569 688 tablet_bootstrap.cc:492] T d9373f55da9c409792ec85c46bc9783e P 36689af90e274323986170dc9d9ce213: Bootstrap starting.
I20260808 11:38:05.059451 683 log.cc:824] T d9373f55da9c409792ec85c46bc9783e P 15e071eb5c6e413d9bc3d993886ed27d: Log is configured to *not* fsync() on all Append() calls
I20260808 11:38:05.060163 684 tablet_bootstrap.cc:654] T d9373f55da9c409792ec85c46bc9783e P 7c07f52546064127addc99be0ee20f3c: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:05.061172 684 log.cc:824] T d9373f55da9c409792ec85c46bc9783e P 7c07f52546064127addc99be0ee20f3c: Log is configured to *not* fsync() on all Append() calls
I20260808 11:38:05.061565 688 tablet_bootstrap.cc:654] T d9373f55da9c409792ec85c46bc9783e P 36689af90e274323986170dc9d9ce213: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:05.062608 683 tablet_bootstrap.cc:492] T d9373f55da9c409792ec85c46bc9783e P 15e071eb5c6e413d9bc3d993886ed27d: No bootstrap required, opened a new log
I20260808 11:38:05.062721 688 log.cc:824] T d9373f55da9c409792ec85c46bc9783e P 36689af90e274323986170dc9d9ce213: Log is configured to *not* fsync() on all Append() calls
I20260808 11:38:05.062924 683 ts_tablet_manager.cc:1397] T d9373f55da9c409792ec85c46bc9783e P 15e071eb5c6e413d9bc3d993886ed27d: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260808 11:38:05.064435 684 tablet_bootstrap.cc:492] T d9373f55da9c409792ec85c46bc9783e P 7c07f52546064127addc99be0ee20f3c: No bootstrap required, opened a new log
I20260808 11:38:05.064705 684 ts_tablet_manager.cc:1397] T d9373f55da9c409792ec85c46bc9783e P 7c07f52546064127addc99be0ee20f3c: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260808 11:38:05.065578 688 tablet_bootstrap.cc:492] T d9373f55da9c409792ec85c46bc9783e P 36689af90e274323986170dc9d9ce213: No bootstrap required, opened a new log
I20260808 11:38:05.065820 688 ts_tablet_manager.cc:1397] T d9373f55da9c409792ec85c46bc9783e P 36689af90e274323986170dc9d9ce213: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 11:38:05.069856 683 raft_consensus.cc:348] T d9373f55da9c409792ec85c46bc9783e P 15e071eb5c6e413d9bc3d993886ed27d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c07f52546064127addc99be0ee20f3c" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 40111 } } peers { permanent_uuid: "15e071eb5c6e413d9bc3d993886ed27d" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 40983 } } peers { permanent_uuid: "36689af90e274323986170dc9d9ce213" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 46561 } }
I20260808 11:38:05.070235 683 raft_consensus.cc:374] T d9373f55da9c409792ec85c46bc9783e P 15e071eb5c6e413d9bc3d993886ed27d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:05.070356 683 raft_consensus.cc:729] T d9373f55da9c409792ec85c46bc9783e P 15e071eb5c6e413d9bc3d993886ed27d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15e071eb5c6e413d9bc3d993886ed27d, State: Initialized, Role: FOLLOWER
I20260808 11:38:05.070993 683 consensus_queue.cc:260] T d9373f55da9c409792ec85c46bc9783e P 15e071eb5c6e413d9bc3d993886ed27d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c07f52546064127addc99be0ee20f3c" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 40111 } } peers { permanent_uuid: "15e071eb5c6e413d9bc3d993886ed27d" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 40983 } } peers { permanent_uuid: "36689af90e274323986170dc9d9ce213" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 46561 } }
I20260808 11:38:05.072427 32706 heartbeater.cc:503] Master 127.24.76.126:41113 was elected leader, sending a full tablet report...
I20260808 11:38:05.073118 683 ts_tablet_manager.cc:1428] T d9373f55da9c409792ec85c46bc9783e P 15e071eb5c6e413d9bc3d993886ed27d: Time spent starting tablet: real 0.010s user 0.009s sys 0.001s
I20260808 11:38:05.074674 684 raft_consensus.cc:348] T d9373f55da9c409792ec85c46bc9783e P 7c07f52546064127addc99be0ee20f3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c07f52546064127addc99be0ee20f3c" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 40111 } } peers { permanent_uuid: "15e071eb5c6e413d9bc3d993886ed27d" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 40983 } } peers { permanent_uuid: "36689af90e274323986170dc9d9ce213" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 46561 } }
I20260808 11:38:05.075075 684 raft_consensus.cc:374] T d9373f55da9c409792ec85c46bc9783e P 7c07f52546064127addc99be0ee20f3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:05.075172 684 raft_consensus.cc:729] T d9373f55da9c409792ec85c46bc9783e P 7c07f52546064127addc99be0ee20f3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c07f52546064127addc99be0ee20f3c, State: Initialized, Role: FOLLOWER
I20260808 11:38:05.075744 684 consensus_queue.cc:260] T d9373f55da9c409792ec85c46bc9783e P 7c07f52546064127addc99be0ee20f3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c07f52546064127addc99be0ee20f3c" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 40111 } } peers { permanent_uuid: "15e071eb5c6e413d9bc3d993886ed27d" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 40983 } } peers { permanent_uuid: "36689af90e274323986170dc9d9ce213" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 46561 } }
I20260808 11:38:05.075881 688 raft_consensus.cc:348] T d9373f55da9c409792ec85c46bc9783e P 36689af90e274323986170dc9d9ce213 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c07f52546064127addc99be0ee20f3c" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 40111 } } peers { permanent_uuid: "15e071eb5c6e413d9bc3d993886ed27d" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 40983 } } peers { permanent_uuid: "36689af90e274323986170dc9d9ce213" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 46561 } }
I20260808 11:38:05.076357 688 raft_consensus.cc:374] T d9373f55da9c409792ec85c46bc9783e P 36689af90e274323986170dc9d9ce213 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:05.076494 688 raft_consensus.cc:729] T d9373f55da9c409792ec85c46bc9783e P 36689af90e274323986170dc9d9ce213 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36689af90e274323986170dc9d9ce213, State: Initialized, Role: FOLLOWER
I20260808 11:38:05.076784 448 heartbeater.cc:503] Master 127.24.76.126:41113 was elected leader, sending a full tablet report...
I20260808 11:38:05.077389 684 ts_tablet_manager.cc:1428] T d9373f55da9c409792ec85c46bc9783e P 7c07f52546064127addc99be0ee20f3c: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
I20260808 11:38:05.078337 688 consensus_queue.cc:260] T d9373f55da9c409792ec85c46bc9783e P 36689af90e274323986170dc9d9ce213 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c07f52546064127addc99be0ee20f3c" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 40111 } } peers { permanent_uuid: "15e071eb5c6e413d9bc3d993886ed27d" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 40983 } } peers { permanent_uuid: "36689af90e274323986170dc9d9ce213" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 46561 } }
I20260808 11:38:05.079412 656 heartbeater.cc:503] Master 127.24.76.126:41113 was elected leader, sending a full tablet report...
I20260808 11:38:05.079824 688 ts_tablet_manager.cc:1428] T d9373f55da9c409792ec85c46bc9783e P 36689af90e274323986170dc9d9ce213: Time spent starting tablet: real 0.014s user 0.010s sys 0.002s
W20260808 11:38:05.120062 32707 tablet.cc:2369] T d9373f55da9c409792ec85c46bc9783e P 15e071eb5c6e413d9bc3d993886ed27d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:38:05.219439 658 tablet.cc:2369] T d9373f55da9c409792ec85c46bc9783e P 36689af90e274323986170dc9d9ce213: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:38:05.224743 694 raft_consensus.cc:482] T d9373f55da9c409792ec85c46bc9783e P 15e071eb5c6e413d9bc3d993886ed27d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:38:05.225049 694 raft_consensus.cc:504] T d9373f55da9c409792ec85c46bc9783e P 15e071eb5c6e413d9bc3d993886ed27d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c07f52546064127addc99be0ee20f3c" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 40111 } } peers { permanent_uuid: "15e071eb5c6e413d9bc3d993886ed27d" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 40983 } } peers { permanent_uuid: "36689af90e274323986170dc9d9ce213" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 46561 } }
I20260808 11:38:05.226722 694 leader_election.cc:288] T d9373f55da9c409792ec85c46bc9783e P 15e071eb5c6e413d9bc3d993886ed27d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c07f52546064127addc99be0ee20f3c (127.24.76.66:40111), 36689af90e274323986170dc9d9ce213 (127.24.76.67:46561)
I20260808 11:38:05.229939 682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:05.227258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:42111 (local address 127.24.76.67:46561)
0808 11:38:05.227411 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:05.227416 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:05.227957 (+ 541us) server_negotiation.cc:407] Connection header received
0808 11:38:05.228127 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:05.228131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:05.228183 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:05.228264 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:05.229216 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:05.229222 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:05.229227 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:38:05.229248 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:05.229289 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:38:05.229305 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:05.229437 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:05.229603 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 11:38:05.229666 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 11:38:05.230162 701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:05.227103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:46561 (local address 127.24.76.65:42111)
0808 11:38:05.227848 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:05.227865 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:05.227998 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:05.228313 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:05.228317 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:05.228341 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:05.228990 (+ 649us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:38:05.228994 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:05.229012 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:38:05.229061 (+ 49us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:05.229455 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:05.229460 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:05.229465 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 11:38:05.229609 (+ 144us) client_negotiation.cc:240] Negotiation successful
0808 11:38:05.229664 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":848,"threads_started":1}
I20260808 11:38:05.230409 676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:05.227204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:52759 (local address 127.24.76.66:40111)
0808 11:38:05.227394 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:05.227401 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:05.228363 (+ 962us) server_negotiation.cc:407] Connection header received
0808 11:38:05.228498 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:05.228504 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:05.228553 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:05.228634 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:05.229470 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:05.229475 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:05.229480 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:38:05.229509 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:05.229549 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:38:05.229565 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:05.229678 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:05.229939 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 11:38:05.230077 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 11:38:05.230557 702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:05.227102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:40111 (local address 127.24.76.65:52759)
0808 11:38:05.228253 (+ 1151us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:05.228267 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:05.228373 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:05.228673 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:05.228677 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:05.228699 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:05.229268 (+ 569us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:38:05.229271 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:05.229285 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:38:05.229325 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:05.229729 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:05.229733 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:05.229739 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 11:38:05.229927 (+ 188us) client_negotiation.cc:240] Negotiation successful
0808 11:38:05.229981 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1050,"mutex_wait_us":26,"thread_start_us":402,"threads_started":1}
I20260808 11:38:05.231283 581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9373f55da9c409792ec85c46bc9783e" candidate_uuid: "15e071eb5c6e413d9bc3d993886ed27d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36689af90e274323986170dc9d9ce213" is_pre_election: true
I20260808 11:38:05.231580 375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9373f55da9c409792ec85c46bc9783e" candidate_uuid: "15e071eb5c6e413d9bc3d993886ed27d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c07f52546064127addc99be0ee20f3c" is_pre_election: true
I20260808 11:38:05.231858 581 raft_consensus.cc:2454] T d9373f55da9c409792ec85c46bc9783e P 36689af90e274323986170dc9d9ce213 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15e071eb5c6e413d9bc3d993886ed27d in term 0.
I20260808 11:38:05.232161 375 raft_consensus.cc:2454] T d9373f55da9c409792ec85c46bc9783e P 7c07f52546064127addc99be0ee20f3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15e071eb5c6e413d9bc3d993886ed27d in term 0.
I20260808 11:38:05.232587 32519 leader_election.cc:302] T d9373f55da9c409792ec85c46bc9783e P 15e071eb5c6e413d9bc3d993886ed27d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15e071eb5c6e413d9bc3d993886ed27d, 36689af90e274323986170dc9d9ce213; no voters:
I20260808 11:38:05.232915 694 raft_consensus.cc:2781] T d9373f55da9c409792ec85c46bc9783e P 15e071eb5c6e413d9bc3d993886ed27d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:38:05.233057 694 raft_consensus.cc:482] T d9373f55da9c409792ec85c46bc9783e P 15e071eb5c6e413d9bc3d993886ed27d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:38:05.233156 694 raft_consensus.cc:3037] T d9373f55da9c409792ec85c46bc9783e P 15e071eb5c6e413d9bc3d993886ed27d [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:05.235425 694 raft_consensus.cc:504] T d9373f55da9c409792ec85c46bc9783e P 15e071eb5c6e413d9bc3d993886ed27d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c07f52546064127addc99be0ee20f3c" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 40111 } } peers { permanent_uuid: "15e071eb5c6e413d9bc3d993886ed27d" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 40983 } } peers { permanent_uuid: "36689af90e274323986170dc9d9ce213" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 46561 } }
I20260808 11:38:05.235962 694 leader_election.cc:288] T d9373f55da9c409792ec85c46bc9783e P 15e071eb5c6e413d9bc3d993886ed27d [CANDIDATE]: Term 1 election: Requested vote from peers 7c07f52546064127addc99be0ee20f3c (127.24.76.66:40111), 36689af90e274323986170dc9d9ce213 (127.24.76.67:46561)
I20260808 11:38:05.236366 375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9373f55da9c409792ec85c46bc9783e" candidate_uuid: "15e071eb5c6e413d9bc3d993886ed27d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c07f52546064127addc99be0ee20f3c"
I20260808 11:38:05.236387 581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9373f55da9c409792ec85c46bc9783e" candidate_uuid: "15e071eb5c6e413d9bc3d993886ed27d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36689af90e274323986170dc9d9ce213"
I20260808 11:38:05.236522 375 raft_consensus.cc:3037] T d9373f55da9c409792ec85c46bc9783e P 7c07f52546064127addc99be0ee20f3c [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:05.236557 581 raft_consensus.cc:3037] T d9373f55da9c409792ec85c46bc9783e P 36689af90e274323986170dc9d9ce213 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:05.238689 581 raft_consensus.cc:2454] T d9373f55da9c409792ec85c46bc9783e P 36689af90e274323986170dc9d9ce213 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15e071eb5c6e413d9bc3d993886ed27d in term 1.
I20260808 11:38:05.238705 375 raft_consensus.cc:2454] T d9373f55da9c409792ec85c46bc9783e P 7c07f52546064127addc99be0ee20f3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15e071eb5c6e413d9bc3d993886ed27d in term 1.
I20260808 11:38:05.239032 32519 leader_election.cc:302] T d9373f55da9c409792ec85c46bc9783e P 15e071eb5c6e413d9bc3d993886ed27d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15e071eb5c6e413d9bc3d993886ed27d, 36689af90e274323986170dc9d9ce213; no voters:
I20260808 11:38:05.239296 694 raft_consensus.cc:2781] T d9373f55da9c409792ec85c46bc9783e P 15e071eb5c6e413d9bc3d993886ed27d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:38:05.239705 694 raft_consensus.cc:686] T d9373f55da9c409792ec85c46bc9783e P 15e071eb5c6e413d9bc3d993886ed27d [term 1 LEADER]: Becoming Leader. State: Replica: 15e071eb5c6e413d9bc3d993886ed27d, State: Running, Role: LEADER
I20260808 11:38:05.240130 694 consensus_queue.cc:237] T d9373f55da9c409792ec85c46bc9783e P 15e071eb5c6e413d9bc3d993886ed27d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c07f52546064127addc99be0ee20f3c" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 40111 } } peers { permanent_uuid: "15e071eb5c6e413d9bc3d993886ed27d" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 40983 } } peers { permanent_uuid: "36689af90e274323986170dc9d9ce213" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 46561 } }
I20260808 11:38:05.244798 32409 catalog_manager.cc:5495] T d9373f55da9c409792ec85c46bc9783e P 15e071eb5c6e413d9bc3d993886ed27d reported cstate change: term changed from 0 to 1, leader changed from <none> to 15e071eb5c6e413d9bc3d993886ed27d (127.24.76.65). New cstate: current_term: 1 leader_uuid: "15e071eb5c6e413d9bc3d993886ed27d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c07f52546064127addc99be0ee20f3c" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 40111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15e071eb5c6e413d9bc3d993886ed27d" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 40983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36689af90e274323986170dc9d9ce213" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 46561 } health_report { overall_health: UNKNOWN } } }
I20260808 11:38:05.274616 680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:05.272678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47348 (local address 127.24.76.65:40983)
0808 11:38:05.272838 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:05.272842 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:05.272980 (+ 138us) server_negotiation.cc:407] Connection header received
0808 11:38:05.273149 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:05.273152 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:05.273231 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:05.273312 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:05.274146 (+ 834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:05.274150 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:05.274153 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:38:05.274168 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:05.274200 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:38:05.274212 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:05.274300 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:05.274361 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 11:38:05.274403 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 11:38:05.282814 581 raft_consensus.cc:1261] T d9373f55da9c409792ec85c46bc9783e P 36689af90e274323986170dc9d9ce213 [term 1 FOLLOWER]: Refusing update from remote peer 15e071eb5c6e413d9bc3d993886ed27d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:38:05.282814 375 raft_consensus.cc:1261] T d9373f55da9c409792ec85c46bc9783e P 7c07f52546064127addc99be0ee20f3c [term 1 FOLLOWER]: Refusing update from remote peer 15e071eb5c6e413d9bc3d993886ed27d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:38:05.283864 694 consensus_queue.cc:1035] T d9373f55da9c409792ec85c46bc9783e P 15e071eb5c6e413d9bc3d993886ed27d [LEADER]: Connected to new peer: Peer: permanent_uuid: "36689af90e274323986170dc9d9ce213" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 46561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:38:05.284157 703 consensus_queue.cc:1035] T d9373f55da9c409792ec85c46bc9783e P 15e071eb5c6e413d9bc3d993886ed27d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c07f52546064127addc99be0ee20f3c" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 40111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 11:38:05.291905 449 tablet.cc:2369] T d9373f55da9c409792ec85c46bc9783e P 7c07f52546064127addc99be0ee20f3c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:38:05.293136 707 mvcc.cc:205] Tried to move back new op lower bound from 7316230493307514880 to 7316230493146984448. Current Snapshot: MvccSnapshot[applied={T|T < 7316230493307514880}]
I20260808 11:38:05.308646 32410 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36208:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:38:05.308866 32410 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36208:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:38:05.312407 32410 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7256926d80b240b18618eea079af9ed4: Sending DeleteTablet for 3 replicas of tablet d9373f55da9c409792ec85c46bc9783e
I20260808 11:38:05.313524 32593 tablet_service.cc:1514] Processing DeleteTablet for tablet d9373f55da9c409792ec85c46bc9783e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:38:05 UTC) from {username='slave'} at 127.0.0.1:47344
I20260808 11:38:05.313514 556 tablet_service.cc:1514] Processing DeleteTablet for tablet d9373f55da9c409792ec85c46bc9783e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:38:05 UTC) from {username='slave'} at 127.0.0.1:41262
I20260808 11:38:05.313520 348 tablet_service.cc:1514] Processing DeleteTablet for tablet d9373f55da9c409792ec85c46bc9783e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:38:05 UTC) from {username='slave'} at 127.0.0.1:49992
I20260808 11:38:05.314512 725 tablet_replica.cc:330] T d9373f55da9c409792ec85c46bc9783e P 7c07f52546064127addc99be0ee20f3c: stopping tablet replica
I20260808 11:38:05.314986 725 raft_consensus.cc:2229] T d9373f55da9c409792ec85c46bc9783e P 7c07f52546064127addc99be0ee20f3c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:38:05.315163 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 32494
I20260808 11:38:05.315555 725 raft_consensus.cc:2258] T d9373f55da9c409792ec85c46bc9783e P 7c07f52546064127addc99be0ee20f3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:38:05.319691 722 tablet_replica.cc:330] T d9373f55da9c409792ec85c46bc9783e P 36689af90e274323986170dc9d9ce213: stopping tablet replica
I20260808 11:38:05.320156 722 raft_consensus.cc:2229] T d9373f55da9c409792ec85c46bc9783e P 36689af90e274323986170dc9d9ce213 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:38:05.320641 725 ts_tablet_manager.cc:1905] T d9373f55da9c409792ec85c46bc9783e P 7c07f52546064127addc99be0ee20f3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:38:05.320786 722 raft_consensus.cc:2258] T d9373f55da9c409792ec85c46bc9783e P 36689af90e274323986170dc9d9ce213 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:38:05.320837 680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:05.316208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47352 (local address 127.24.76.65:40983)
0808 11:38:05.316371 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:05.316376 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:05.318372 (+ 1996us) server_negotiation.cc:407] Connection header received
0808 11:38:05.318646 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:05.318652 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:05.318715 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:05.318801 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:05.319996 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:05.320002 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:05.320007 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:38:05.320027 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:05.320064 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:38:05.320080 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:05.320178 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:05.320483 (+ 305us) server_negotiation.cc:299] Negotiation successful
0808 11:38:05.320547 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 11:38:05.321820 32696 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:05.323258 722 ts_tablet_manager.cc:1905] T d9373f55da9c409792ec85c46bc9783e P 36689af90e274323986170dc9d9ce213: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:38:05.328083 725 ts_tablet_manager.cc:1918] T d9373f55da9c409792ec85c46bc9783e P 7c07f52546064127addc99be0ee20f3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:38:05.328262 725 log.cc:1196] T d9373f55da9c409792ec85c46bc9783e P 7c07f52546064127addc99be0ee20f3c: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-1/wal/wals/d9373f55da9c409792ec85c46bc9783e
I20260808 11:38:05.328771 725 ts_tablet_manager.cc:1939] T d9373f55da9c409792ec85c46bc9783e P 7c07f52546064127addc99be0ee20f3c: Deleting consensus metadata
I20260808 11:38:05.329449 722 ts_tablet_manager.cc:1918] T d9373f55da9c409792ec85c46bc9783e P 36689af90e274323986170dc9d9ce213: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:38:05.329592 722 log.cc:1196] T d9373f55da9c409792ec85c46bc9783e P 36689af90e274323986170dc9d9ce213: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786189042529715-24881-0/minicluster-data/ts-2/wal/wals/d9373f55da9c409792ec85c46bc9783e
I20260808 11:38:05.330051 722 ts_tablet_manager.cc:1939] T d9373f55da9c409792ec85c46bc9783e P 36689af90e274323986170dc9d9ce213: Deleting consensus metadata
I20260808 11:38:05.330646 32392 catalog_manager.cc:4841] TS 7c07f52546064127addc99be0ee20f3c (127.24.76.66:40111): tablet d9373f55da9c409792ec85c46bc9783e (table test-table [id=7bca775500164b5495b40c7af343928a]) successfully deleted
I20260808 11:38:05.331545 32390 catalog_manager.cc:4841] TS 36689af90e274323986170dc9d9ce213 (127.24.76.67:46561): tablet d9373f55da9c409792ec85c46bc9783e (table test-table [id=7bca775500164b5495b40c7af343928a]) successfully deleted
I20260808 11:38:05.414456 723 tablet_replica.cc:330] T d9373f55da9c409792ec85c46bc9783e P 15e071eb5c6e413d9bc3d993886ed27d: stopping tablet replica
I20260808 11:38:05.415061 723 raft_consensus.cc:2229] T d9373f55da9c409792ec85c46bc9783e P 15e071eb5c6e413d9bc3d993886ed27d [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:38:05.415318 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 32494
I20260808 11:38:05.439659 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 32713
W20260808 11:38:05.439448 32388 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.76.65:40983 (error 108)
I20260808 11:38:05.441530 677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:05.441043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:40983 (local address 127.0.0.1:47364)
0808 11:38:05.441196 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:05.441368 (+ 172us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:40983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 11:38:05.441815 32388 catalog_manager.cc:4584] TS 15e071eb5c6e413d9bc3d993886ed27d (127.24.76.65:40983): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d9373f55da9c409792ec85c46bc9783e: Network error: Client connection negotiation failed: client connection to 127.24.76.65:40983: connect: Connection refused (error 111)
I20260808 11:38:05.442637 676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:05.440591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49996 (local address 127.24.76.66:40111)
0808 11:38:05.440761 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:05.440767 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:05.440787 (+ 20us) server_negotiation.cc:407] Connection header received
0808 11:38:05.440936 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:05.440940 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:05.441030 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:05.441118 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:05.442036 (+ 918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:05.442042 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:05.442047 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:38:05.442066 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:05.442106 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:38:05.442123 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:05.442259 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:05.442330 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 11:38:05.442378 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 11:38:05.443370 441 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:05.495688 677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:05.495195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:40983 (local address 127.0.0.1:47378)
0808 11:38:05.495429 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:05.495533 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:40983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260808 11:38:05.496661 677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:05.496321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:40983 (local address 127.0.0.1:47392)
0808 11:38:05.496472 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:05.496542 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:40983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 11:38:05.530229 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 32713
I20260808 11:38:05.554090 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 452
I20260808 11:38:05.556839 677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:05.556379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:40983 (local address 127.0.0.1:47400)
0808 11:38:05.556548 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:05.556643 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:40983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 11:38:05.557605 682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:05.554920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41278 (local address 127.24.76.67:46561)
0808 11:38:05.555457 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:05.555463 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:05.555483 (+ 20us) server_negotiation.cc:407] Connection header received
0808 11:38:05.555551 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:05.555556 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:05.555657 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:05.555766 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:05.556766 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:05.556773 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:05.556778 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:38:05.556799 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:05.556842 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:38:05.556859 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:05.556996 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:05.557238 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 11:38:05.557307 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402}
I20260808 11:38:05.558225 677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:05.557544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:40983 (local address 127.0.0.1:47412)
0808 11:38:05.557979 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:05.558078 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:40983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305}
I20260808 11:38:05.558266 646 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:05.638758 677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:05.638304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:40983 (local address 127.0.0.1:47426)
0808 11:38:05.638514 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:05.638606 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:40983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260808 11:38:05.639822 677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:05.639444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:40983 (local address 127.0.0.1:47432)
0808 11:38:05.639632 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:05.639703 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:40983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260808 11:38:05.654980 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 452
I20260808 11:38:05.680039 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 32367
I20260808 11:38:05.681217 32459 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:05.787683 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 32367
2026-08-08T11:38:05Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2655 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 11:38:05.834960 24881 mini_oidc.cc:152] Starting JWKS server
I20260808 11:38:05.838035 24881 webserver.cc:460] Webserver started at https://127.0.0.1:38237/ using document root <none> and password file <none>
I20260808 11:38:05.838451 24881 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 11:38:05.839082 24881 webserver.cc:460] Webserver started at http://127.0.0.1:38151/ using document root <none> and password file <none>
2026-08-08T11:38:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:38:05Z Disabled control of system clock
I20260808 11:38:05.871285 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:44867
--webserver_interface=127.24.76.126
--webserver_port=0
--builtin_ntp_servers=127.24.76.84:38387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:44867
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38237/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189042529715-24881-0/testchainca.cert with env {}
W20260808 11:38:06.187530 746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:06.187817 746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:06.187949 746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:06.196007 746 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:38:06.196120 746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:06.196182 746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:06.196259 746 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:38:06.196318 746 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 11:38:06.196398 746 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 11:38:06.196452 746 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38237/jwks.json
I20260808 11:38:06.206303 746 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:38387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:44867
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:44867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38237/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189042529715-24881-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:06.207612 746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:06.208848 746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:06.219899 757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:06.219980 754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:06.219941 755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:06.220664 746 server_base.cc:1034] running on GCE node
I20260808 11:38:06.221383 746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:06.222690 746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:06.223886 746 hybrid_clock.cc:648] HybridClock initialized: now 1786189086223847 us; error 32 us; skew 500 ppm
I20260808 11:38:06.224218 746 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:38:06.226300 746 webserver.cc:460] Webserver started at http://127.24.76.126:44997/ using document root <none> and password file <none>
I20260808 11:38:06.226925 746 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:06.227041 746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:06.227335 746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:06.230243 746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "17314fea5a164199a3bd2c0259f93617"
format_stamp: "Formatted at 2026-08-08 11:38:06 on dist-test-slave-tc2s"
server_key: "1d86380b5f36c6d898f5f802e279064e"
server_key_iv: "5f72eb1824933d3fc5e8f59247790050"
server_key_version: "encryptionkey@0"
I20260808 11:38:06.231055 746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "17314fea5a164199a3bd2c0259f93617"
format_stamp: "Formatted at 2026-08-08 11:38:06 on dist-test-slave-tc2s"
server_key: "1d86380b5f36c6d898f5f802e279064e"
server_key_iv: "5f72eb1824933d3fc5e8f59247790050"
server_key_version: "encryptionkey@0"
I20260808 11:38:06.235950 746 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 11:38:06.239161 768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:06.240376 746 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:38:06.240535 746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "17314fea5a164199a3bd2c0259f93617"
format_stamp: "Formatted at 2026-08-08 11:38:06 on dist-test-slave-tc2s"
server_key: "1d86380b5f36c6d898f5f802e279064e"
server_key_iv: "5f72eb1824933d3fc5e8f59247790050"
server_key_version: "encryptionkey@0"
I20260808 11:38:06.240733 746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:06.288339 746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:06.289191 746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:06.289482 746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:06.306206 746 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:44867
I20260808 11:38:06.306226 845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:44867 every 8 connection(s)
I20260808 11:38:06.307444 746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:38:06.308501 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 746
I20260808 11:38:06.308706 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:38:06.309310 24881 external_mini_cluster.cc:1428] Setting key 37ac1221751cecf2b2dfd228c8532c64
I20260808 11:38:06.311810 846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:06.325034 854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:06.311212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51876 (local address 127.24.76.126:44867)
0808 11:38:06.312636 (+ 1424us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:06.312650 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:06.312700 (+ 50us) server_negotiation.cc:407] Connection header received
0808 11:38:06.314852 (+ 2152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:06.314879 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:06.315475 (+ 596us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:06.315904 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:06.316402 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:06.318415 (+ 2013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:06.319758 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:06.320742 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:06.320961 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:06.322279 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:06.322290 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:06.322307 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:38:06.323036 (+ 729us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:06.323130 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:38:06.323157 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:06.323273 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:06.323509 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 11:38:06.323835 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":918,"thread_start_us":685,"threads_started":1}
I20260808 11:38:06.326040 846 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17314fea5a164199a3bd2c0259f93617: Bootstrap starting.
I20260808 11:38:06.329039 24881 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:38:06.329573 846 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17314fea5a164199a3bd2c0259f93617: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:06.330549 846 log.cc:824] T 00000000000000000000000000000000 P 17314fea5a164199a3bd2c0259f93617: Log is configured to *not* fsync() on all Append() calls
I20260808 11:38:06.333338 846 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17314fea5a164199a3bd2c0259f93617: No bootstrap required, opened a new log
I20260808 11:38:06.340705 846 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17314fea5a164199a3bd2c0259f93617 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17314fea5a164199a3bd2c0259f93617" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44867 } }
I20260808 11:38:06.341077 846 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17314fea5a164199a3bd2c0259f93617 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:06.341195 846 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17314fea5a164199a3bd2c0259f93617 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17314fea5a164199a3bd2c0259f93617, State: Initialized, Role: FOLLOWER
I20260808 11:38:06.341809 846 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17314fea5a164199a3bd2c0259f93617 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17314fea5a164199a3bd2c0259f93617" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44867 } }
I20260808 11:38:06.342072 846 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17314fea5a164199a3bd2c0259f93617 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:38:06.342176 846 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17314fea5a164199a3bd2c0259f93617 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:38:06.342307 846 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17314fea5a164199a3bd2c0259f93617 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:06.344439 846 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17314fea5a164199a3bd2c0259f93617 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17314fea5a164199a3bd2c0259f93617" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44867 } }
I20260808 11:38:06.344905 846 leader_election.cc:302] T 00000000000000000000000000000000 P 17314fea5a164199a3bd2c0259f93617 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17314fea5a164199a3bd2c0259f93617; no voters:
I20260808 11:38:06.345414 846 leader_election.cc:288] T 00000000000000000000000000000000 P 17314fea5a164199a3bd2c0259f93617 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:38:06.345547 861 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17314fea5a164199a3bd2c0259f93617 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:38:06.346206 861 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17314fea5a164199a3bd2c0259f93617 [term 1 LEADER]: Becoming Leader. State: Replica: 17314fea5a164199a3bd2c0259f93617, State: Running, Role: LEADER
I20260808 11:38:06.346712 861 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17314fea5a164199a3bd2c0259f93617 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17314fea5a164199a3bd2c0259f93617" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44867 } }
I20260808 11:38:06.347306 846 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17314fea5a164199a3bd2c0259f93617 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:38:06.350265 864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17314fea5a164199a3bd2c0259f93617 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17314fea5a164199a3bd2c0259f93617. Latest consensus state: current_term: 1 leader_uuid: "17314fea5a164199a3bd2c0259f93617" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17314fea5a164199a3bd2c0259f93617" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44867 } } }
I20260808 11:38:06.350718 864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17314fea5a164199a3bd2c0259f93617 [sys.catalog]: This master's current role is: LEADER
I20260808 11:38:06.350940 863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17314fea5a164199a3bd2c0259f93617 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17314fea5a164199a3bd2c0259f93617" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17314fea5a164199a3bd2c0259f93617" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44867 } } }
I20260808 11:38:06.351461 863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17314fea5a164199a3bd2c0259f93617 [sys.catalog]: This master's current role is: LEADER
I20260808 11:38:06.352793 878 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:38:06.357786 878 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:38:06.365872 878 catalog_manager.cc:1347] Generated new cluster ID: 049385f8df294c79a1f7dd72fe8fcfcc
I20260808 11:38:06.366014 878 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:38:06.392202 878 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:38:06.393443 878 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:38:06.410616 878 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17314fea5a164199a3bd2c0259f93617: Generated new TSK 0
I20260808 11:38:06.411473 878 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 11:38:06.455617 903 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 11:38:06.455832 903 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 11:38:06.458050 854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:06.436306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51880 (local address 127.24.76.126:44867)
0808 11:38:06.436454 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:06.436459 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:06.436995 (+ 536us) server_negotiation.cc:407] Connection header received
0808 11:38:06.437335 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:06.437340 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:06.437405 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:06.437534 (+ 129us) server_negotiation.cc:226] Negotiated authn=JWT
0808 11:38:06.438102 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:06.438887 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:06.440379 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:06.440815 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:06.440923 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:06.441413 (+ 490us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 11:38:06.457368 (+ 15955us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38237/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 11:38:06.457698 (+ 330us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51880: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38237/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 11:38:06.458179 854 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51880: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38237/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 11:38:06.459674 902 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:06.436282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44867 (local address 127.0.0.1:51880)
0808 11:38:06.436884 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:06.436899 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:06.437067 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:06.437584 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:06.437590 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:06.437602 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0808 11:38:06.437960 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:06.437972 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:06.439033 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:06.439038 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:06.440231 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:06.440244 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:06.440971 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:06.440977 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:06.441116 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:06.441173 (+ 57us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 11:38:06.457630 (+ 16457us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38237/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 11:38:06.457967 (+ 337us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.76.126:44867: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38237/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":314,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.24.76.126:44867: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38237/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 11:38:06.461241 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 746
I20260808 11:38:06.462195 839 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:06.558140 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 746
2026-08-08T11:38:06Z chronyd exiting
I20260808 11:38:06.616456 24881 test_util.cc:175] -----------------------------------------------
I20260808 11:38:06.616602 24881 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786189042529715-24881-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (786 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 11:38:06.620492 24881 mini_oidc.cc:152] Starting JWKS server
I20260808 11:38:06.622465 24881 webserver.cc:460] Webserver started at https://127.0.0.1:42505/ using document root <none> and password file <none>
I20260808 11:38:06.622695 24881 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 11:38:06.623283 24881 webserver.cc:460] Webserver started at http://127.0.0.1:37563/ using document root <none> and password file <none>
2026-08-08T11:38:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:38:06Z Disabled control of system clock
I20260808 11:38:06.654940 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:38135
--webserver_interface=127.24.76.126
--webserver_port=0
--builtin_ntp_servers=127.24.76.84:40989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:38135
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42505/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189042529715-24881-0/testchainca.cert with env {}
W20260808 11:38:06.969350 913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:06.969633 913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:06.969769 913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:06.976797 913 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:38:06.976907 913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:06.976972 913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:06.977062 913 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:38:06.977126 913 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 11:38:06.977210 913 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 11:38:06.977264 913 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42505/jwks.json
I20260808 11:38:06.987162 913 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:40989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:38135
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:38135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42505/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189042529715-24881-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:06.988364 913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:06.989641 913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:07.000767 919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:07.000837 920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:07.000806 922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:07.001680 913 server_base.cc:1034] running on GCE node
I20260808 11:38:07.002470 913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:07.003746 913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:07.004944 913 hybrid_clock.cc:648] HybridClock initialized: now 1786189087004900 us; error 41 us; skew 500 ppm
I20260808 11:38:07.005276 913 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:38:07.007432 913 webserver.cc:460] Webserver started at http://127.24.76.126:37945/ using document root <none> and password file <none>
I20260808 11:38:07.008092 913 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:07.008200 913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:07.008522 913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:07.011399 913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "aab402edf7cc4192be83f332e61a9500"
format_stamp: "Formatted at 2026-08-08 11:38:07 on dist-test-slave-tc2s"
server_key: "12cb0868078a0c302cafc85d55e87f84"
server_key_iv: "c2874da8128e4871c6b0c130d8459ea3"
server_key_version: "encryptionkey@0"
I20260808 11:38:07.012177 913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "aab402edf7cc4192be83f332e61a9500"
format_stamp: "Formatted at 2026-08-08 11:38:07 on dist-test-slave-tc2s"
server_key: "12cb0868078a0c302cafc85d55e87f84"
server_key_iv: "c2874da8128e4871c6b0c130d8459ea3"
server_key_version: "encryptionkey@0"
I20260808 11:38:07.017163 913 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 11:38:07.020440 933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:07.021692 913 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 11:38:07.021996 913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "aab402edf7cc4192be83f332e61a9500"
format_stamp: "Formatted at 2026-08-08 11:38:07 on dist-test-slave-tc2s"
server_key: "12cb0868078a0c302cafc85d55e87f84"
server_key_iv: "c2874da8128e4871c6b0c130d8459ea3"
server_key_version: "encryptionkey@0"
I20260808 11:38:07.022219 913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:07.047428 913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:07.048163 913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:07.048447 913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:07.065619 913 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:38135
I20260808 11:38:07.065639 1005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:38135 every 8 connection(s)
I20260808 11:38:07.066872 913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:38:07.070626 1006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:07.071142 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 913
I20260808 11:38:07.071311 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:38:07.071936 24881 external_mini_cluster.cc:1428] Setting key 38e122422da0261a0685e2777fc255ae
I20260808 11:38:07.084648 1006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aab402edf7cc4192be83f332e61a9500: Bootstrap starting.
I20260808 11:38:07.086517 1011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:07.073765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36966 (local address 127.24.76.126:38135)
0808 11:38:07.074861 (+ 1096us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:07.074874 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:07.074936 (+ 62us) server_negotiation.cc:407] Connection header received
0808 11:38:07.076923 (+ 1987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:07.076953 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:07.077513 (+ 560us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:07.077967 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:07.078463 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:07.080300 (+ 1837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:07.081515 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:07.082588 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:07.082742 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:07.083778 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:07.083789 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:07.083806 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:38:07.084515 (+ 709us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:07.084615 (+ 100us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:38:07.084646 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:07.084772 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:07.084938 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 11:38:07.085236 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":356,"threads_started":1}
I20260808 11:38:07.088153 1006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aab402edf7cc4192be83f332e61a9500: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:07.089255 1006 log.cc:824] T 00000000000000000000000000000000 P aab402edf7cc4192be83f332e61a9500: Log is configured to *not* fsync() on all Append() calls
I20260808 11:38:07.090065 24881 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:38:07.092219 1006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aab402edf7cc4192be83f332e61a9500: No bootstrap required, opened a new log
I20260808 11:38:07.099592 1006 raft_consensus.cc:348] T 00000000000000000000000000000000 P aab402edf7cc4192be83f332e61a9500 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aab402edf7cc4192be83f332e61a9500" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 38135 } }
I20260808 11:38:07.099907 1006 raft_consensus.cc:374] T 00000000000000000000000000000000 P aab402edf7cc4192be83f332e61a9500 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:07.099978 1006 raft_consensus.cc:729] T 00000000000000000000000000000000 P aab402edf7cc4192be83f332e61a9500 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aab402edf7cc4192be83f332e61a9500, State: Initialized, Role: FOLLOWER
I20260808 11:38:07.100521 1006 consensus_queue.cc:260] T 00000000000000000000000000000000 P aab402edf7cc4192be83f332e61a9500 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aab402edf7cc4192be83f332e61a9500" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 38135 } }
I20260808 11:38:07.100749 1006 raft_consensus.cc:388] T 00000000000000000000000000000000 P aab402edf7cc4192be83f332e61a9500 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:38:07.100868 1006 raft_consensus.cc:482] T 00000000000000000000000000000000 P aab402edf7cc4192be83f332e61a9500 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:38:07.101012 1006 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aab402edf7cc4192be83f332e61a9500 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:07.103081 1006 raft_consensus.cc:504] T 00000000000000000000000000000000 P aab402edf7cc4192be83f332e61a9500 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aab402edf7cc4192be83f332e61a9500" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 38135 } }
I20260808 11:38:07.103536 1006 leader_election.cc:302] T 00000000000000000000000000000000 P aab402edf7cc4192be83f332e61a9500 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aab402edf7cc4192be83f332e61a9500; no voters:
I20260808 11:38:07.104027 1006 leader_election.cc:288] T 00000000000000000000000000000000 P aab402edf7cc4192be83f332e61a9500 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:38:07.104153 1015 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aab402edf7cc4192be83f332e61a9500 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:38:07.104835 1015 raft_consensus.cc:686] T 00000000000000000000000000000000 P aab402edf7cc4192be83f332e61a9500 [term 1 LEADER]: Becoming Leader. State: Replica: aab402edf7cc4192be83f332e61a9500, State: Running, Role: LEADER
I20260808 11:38:07.105329 1015 consensus_queue.cc:237] T 00000000000000000000000000000000 P aab402edf7cc4192be83f332e61a9500 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aab402edf7cc4192be83f332e61a9500" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 38135 } }
I20260808 11:38:07.105851 1006 sys_catalog.cc:564] T 00000000000000000000000000000000 P aab402edf7cc4192be83f332e61a9500 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:38:07.108105 1019 sys_catalog.cc:455] T 00000000000000000000000000000000 P aab402edf7cc4192be83f332e61a9500 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aab402edf7cc4192be83f332e61a9500. Latest consensus state: current_term: 1 leader_uuid: "aab402edf7cc4192be83f332e61a9500" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aab402edf7cc4192be83f332e61a9500" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 38135 } } }
I20260808 11:38:07.108075 1018 sys_catalog.cc:455] T 00000000000000000000000000000000 P aab402edf7cc4192be83f332e61a9500 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aab402edf7cc4192be83f332e61a9500" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aab402edf7cc4192be83f332e61a9500" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 38135 } } }
I20260808 11:38:07.108593 1019 sys_catalog.cc:458] T 00000000000000000000000000000000 P aab402edf7cc4192be83f332e61a9500 [sys.catalog]: This master's current role is: LEADER
I20260808 11:38:07.108651 1018 sys_catalog.cc:458] T 00000000000000000000000000000000 P aab402edf7cc4192be83f332e61a9500 [sys.catalog]: This master's current role is: LEADER
I20260808 11:38:07.111368 1028 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:38:07.116341 1028 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:38:07.124925 1028 catalog_manager.cc:1347] Generated new cluster ID: 6931e33c27c841e4be08a226cc6a61fd
I20260808 11:38:07.125016 1028 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:38:07.151909 1028 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:38:07.153142 1028 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:38:07.160233 1028 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aab402edf7cc4192be83f332e61a9500: Generated new TSK 0
I20260808 11:38:07.160962 1028 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 11:38:07.197183 1057 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 11:38:07.197295 1057 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 11:38:07.198401 1011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:07.178423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36972 (local address 127.24.76.126:38135)
0808 11:38:07.178611 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:07.178619 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:07.179113 (+ 494us) server_negotiation.cc:407] Connection header received
0808 11:38:07.179296 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:07.179302 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:07.179400 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:07.179560 (+ 160us) server_negotiation.cc:226] Negotiated authn=JWT
0808 11:38:07.179950 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:07.181057 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:07.182382 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:07.183066 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:07.183187 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:07.183631 (+ 444us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 11:38:07.197799 (+ 14168us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42505/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 11:38:07.198172 (+ 373us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36972: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42505/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":60}
W20260808 11:38:07.198514 1011 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36972: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42505/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 11:38:07.198704 1052 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:07.178421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:38135 (local address 127.0.0.1:36972)
0808 11:38:07.179026 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:07.179041 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:07.179151 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:07.179553 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:07.179557 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:07.179565 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0808 11:38:07.179807 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:07.179815 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:07.181168 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:07.181171 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:07.182252 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:07.182264 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:07.183216 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:07.183220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:07.183352 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:07.183383 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 11:38:07.198047 (+ 14664us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42505/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 11:38:07.198314 (+ 267us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.76.126:38135: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42505/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":314,"threads_started":1}
I20260808 11:38:07.200181 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 913
I20260808 11:38:07.201327 1000 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:07.293080 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 913
2026-08-08T11:38:07Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (736 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 11:38:07.356493 24881 mini_oidc.cc:152] Starting JWKS server
I20260808 11:38:07.358430 24881 webserver.cc:460] Webserver started at https://127.0.0.1:37495/ using document root <none> and password file <none>
I20260808 11:38:07.358652 24881 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 11:38:07.359284 24881 webserver.cc:460] Webserver started at http://127.0.0.1:45861/ using document root <none> and password file <none>
2026-08-08T11:38:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:38:07Z Disabled control of system clock
I20260808 11:38:07.391335 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:39785
--webserver_interface=127.24.76.126
--webserver_port=0
--builtin_ntp_servers=127.24.76.84:39435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:39785
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37495/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 11:38:07.712635 1065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:07.712927 1065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:07.713052 1065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:07.719905 1065 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:38:07.719997 1065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:07.720041 1065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:07.720096 1065 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:38:07.720139 1065 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 11:38:07.720197 1065 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 11:38:07.720237 1065 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37495/jwks.json
I20260808 11:38:07.729421 1065 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:39435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:39785
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:39785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37495/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:07.730571 1065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:07.732224 1065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:07.743243 1072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:07.743280 1073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:07.743288 1078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:07.743908 1065 server_base.cc:1034] running on GCE node
I20260808 11:38:07.744580 1065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:07.745821 1065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:07.747014 1065 hybrid_clock.cc:648] HybridClock initialized: now 1786189087746958 us; error 45 us; skew 500 ppm
I20260808 11:38:07.747321 1065 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:38:07.749414 1065 webserver.cc:460] Webserver started at http://127.24.76.126:37101/ using document root <none> and password file <none>
I20260808 11:38:07.750093 1065 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:07.750185 1065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:07.750485 1065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:07.753271 1065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "a8d186b266b04a04b168d8806f06123f"
format_stamp: "Formatted at 2026-08-08 11:38:07 on dist-test-slave-tc2s"
server_key: "a3fa79c91f02312501ebe8c6a5c2b1f0"
server_key_iv: "694bb96510654c4b7623ab20330d4502"
server_key_version: "encryptionkey@0"
I20260808 11:38:07.754077 1065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "a8d186b266b04a04b168d8806f06123f"
format_stamp: "Formatted at 2026-08-08 11:38:07 on dist-test-slave-tc2s"
server_key: "a3fa79c91f02312501ebe8c6a5c2b1f0"
server_key_iv: "694bb96510654c4b7623ab20330d4502"
server_key_version: "encryptionkey@0"
I20260808 11:38:07.758941 1065 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260808 11:38:07.762156 1084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:07.763351 1065 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 11:38:07.763510 1065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "a8d186b266b04a04b168d8806f06123f"
format_stamp: "Formatted at 2026-08-08 11:38:07 on dist-test-slave-tc2s"
server_key: "a3fa79c91f02312501ebe8c6a5c2b1f0"
server_key_iv: "694bb96510654c4b7623ab20330d4502"
server_key_version: "encryptionkey@0"
I20260808 11:38:07.763695 1065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:07.776748 1065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:07.777446 1065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:07.777716 1065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:07.794361 1065 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:39785
I20260808 11:38:07.794400 1158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:39785 every 8 connection(s)
I20260808 11:38:07.795544 1065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:38:07.798218 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 1065
I20260808 11:38:07.798444 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:38:07.799239 24881 external_mini_cluster.cc:1428] Setting key 89d053e335281b0f2bc1c2ec8fe89bda
I20260808 11:38:07.799490 1159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:07.812403 1165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:07.801003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59100 (local address 127.24.76.126:39785)
0808 11:38:07.801959 (+ 956us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:07.801970 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:07.802017 (+ 47us) server_negotiation.cc:407] Connection header received
0808 11:38:07.803863 (+ 1846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:07.803886 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:07.804329 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:07.804665 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:07.805224 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:07.806623 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:07.808340 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:07.808979 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:07.809107 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:07.810286 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:07.810294 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:07.810309 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:38:07.810844 (+ 535us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:07.810917 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:38:07.810938 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:07.811029 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:07.811224 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 11:38:07.811522 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":345,"threads_started":1}
I20260808 11:38:07.812597 1159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8d186b266b04a04b168d8806f06123f: Bootstrap starting.
I20260808 11:38:07.816012 24881 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:38:07.816088 1159 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8d186b266b04a04b168d8806f06123f: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:07.817027 1159 log.cc:824] T 00000000000000000000000000000000 P a8d186b266b04a04b168d8806f06123f: Log is configured to *not* fsync() on all Append() calls
I20260808 11:38:07.820026 1159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8d186b266b04a04b168d8806f06123f: No bootstrap required, opened a new log
I20260808 11:38:07.827590 1159 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8d186b266b04a04b168d8806f06123f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8d186b266b04a04b168d8806f06123f" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39785 } }
I20260808 11:38:07.827961 1159 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8d186b266b04a04b168d8806f06123f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:07.828048 1159 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8d186b266b04a04b168d8806f06123f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8d186b266b04a04b168d8806f06123f, State: Initialized, Role: FOLLOWER
I20260808 11:38:07.828567 1159 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8d186b266b04a04b168d8806f06123f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8d186b266b04a04b168d8806f06123f" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39785 } }
I20260808 11:38:07.828794 1159 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8d186b266b04a04b168d8806f06123f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:38:07.828915 1159 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8d186b266b04a04b168d8806f06123f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:38:07.829066 1159 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8d186b266b04a04b168d8806f06123f [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:07.831197 1159 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8d186b266b04a04b168d8806f06123f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8d186b266b04a04b168d8806f06123f" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39785 } }
I20260808 11:38:07.831674 1159 leader_election.cc:302] T 00000000000000000000000000000000 P a8d186b266b04a04b168d8806f06123f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8d186b266b04a04b168d8806f06123f; no voters:
I20260808 11:38:07.832222 1159 leader_election.cc:288] T 00000000000000000000000000000000 P a8d186b266b04a04b168d8806f06123f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:38:07.832361 1169 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8d186b266b04a04b168d8806f06123f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:38:07.832978 1169 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8d186b266b04a04b168d8806f06123f [term 1 LEADER]: Becoming Leader. State: Replica: a8d186b266b04a04b168d8806f06123f, State: Running, Role: LEADER
I20260808 11:38:07.833505 1169 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8d186b266b04a04b168d8806f06123f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8d186b266b04a04b168d8806f06123f" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39785 } }
I20260808 11:38:07.834363 1159 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8d186b266b04a04b168d8806f06123f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:38:07.836362 1171 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8d186b266b04a04b168d8806f06123f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8d186b266b04a04b168d8806f06123f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8d186b266b04a04b168d8806f06123f" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39785 } } }
I20260808 11:38:07.836795 1171 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8d186b266b04a04b168d8806f06123f [sys.catalog]: This master's current role is: LEADER
I20260808 11:38:07.837230 1173 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8d186b266b04a04b168d8806f06123f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8d186b266b04a04b168d8806f06123f. Latest consensus state: current_term: 1 leader_uuid: "a8d186b266b04a04b168d8806f06123f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8d186b266b04a04b168d8806f06123f" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39785 } } }
I20260808 11:38:07.837575 1173 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8d186b266b04a04b168d8806f06123f [sys.catalog]: This master's current role is: LEADER
I20260808 11:38:07.839138 1182 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:38:07.844208 1182 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:38:07.852698 1182 catalog_manager.cc:1347] Generated new cluster ID: 18c13d4092c64e618a714577739ba9d3
I20260808 11:38:07.852798 1182 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:38:07.868352 1182 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:38:07.870044 1182 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:38:07.880050 1182 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8d186b266b04a04b168d8806f06123f: Generated new TSK 0
I20260808 11:38:07.880985 1182 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:38:07.906596 1165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:07.898582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59116 (local address 127.24.76.126:39785)
0808 11:38:07.898766 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:07.898772 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:07.899119 (+ 347us) server_negotiation.cc:407] Connection header received
0808 11:38:07.899309 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:07.899315 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:07.899409 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:07.899576 (+ 167us) server_negotiation.cc:226] Negotiated authn=JWT
0808 11:38:07.899968 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:07.901063 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:07.902403 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:07.903102 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:07.903236 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:07.903687 (+ 451us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 11:38:07.905960 (+ 2273us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37495/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 11:38:07.906280 (+ 320us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59116: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37495/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":67}
W20260808 11:38:07.906729 1165 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59116: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37495/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 11:38:07.908296 1205 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:07.898456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:39785 (local address 127.0.0.1:59116)
0808 11:38:07.899021 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:07.899036 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:07.899149 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:07.899571 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:07.899574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:07.899583 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0808 11:38:07.899825 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:07.899832 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:07.901180 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:07.901184 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:07.902270 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:07.902283 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:07.903269 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:07.903273 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:07.903403 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:07.903434 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 11:38:07.906163 (+ 2729us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37495/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 11:38:07.907876 (+ 1713us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.76.126:39785: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37495/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":1427,"thread_start_us":302,"threads_started":1}
I20260808 11:38:07.910240 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 1065
I20260808 11:38:07.911295 1154 generic_service.cc:196] Checking for leaks (request via RPC)
E20260808 11:38:07.917752 1206 webserver.cc:533] Webserver: sslize failed:
E20260808 11:38:07.917841 1206 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 11:38:07.999948 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 1065
2026-08-08T11:38:08Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (702 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T11:38:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:38:08Z Disabled control of system clock
I20260808 11:38:08.090307 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:32925
--webserver_interface=127.24.76.126
--webserver_port=0
--builtin_ntp_servers=127.24.76.84:42005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:32925
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189042529715-24881-0/testchainca.cert with env {}
W20260808 11:38:08.409876 1212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:08.410212 1212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:08.410317 1212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:08.417429 1212 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:38:08.417531 1212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:08.417574 1212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:08.417630 1212 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:38:08.417673 1212 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 11:38:08.417738 1212 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 11:38:08.417778 1212 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 11:38:08.427903 1212 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:42005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:32925
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:32925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189042529715-24881-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:08.429266 1212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:08.430579 1212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:08.441797 1219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:08.441916 1221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:08.441926 1224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:08.442643 1212 server_base.cc:1034] running on GCE node
I20260808 11:38:08.443364 1212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:08.444626 1212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:08.445822 1212 hybrid_clock.cc:648] HybridClock initialized: now 1786189088445779 us; error 48 us; skew 500 ppm
I20260808 11:38:08.446161 1212 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:38:08.448334 1212 webserver.cc:460] Webserver started at http://127.24.76.126:42259/ using document root <none> and password file <none>
I20260808 11:38:08.448966 1212 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:08.449069 1212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:08.449352 1212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:08.452126 1212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "5a728eff91e74c46b6128995076f1992"
format_stamp: "Formatted at 2026-08-08 11:38:08 on dist-test-slave-tc2s"
server_key: "a68421a0916298b81e39d7fd0eeba449"
server_key_iv: "3b8ae9fe97eca8d9a52364dc4eaa0829"
server_key_version: "encryptionkey@0"
I20260808 11:38:08.453043 1212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "5a728eff91e74c46b6128995076f1992"
format_stamp: "Formatted at 2026-08-08 11:38:08 on dist-test-slave-tc2s"
server_key: "a68421a0916298b81e39d7fd0eeba449"
server_key_iv: "3b8ae9fe97eca8d9a52364dc4eaa0829"
server_key_version: "encryptionkey@0"
I20260808 11:38:08.457858 1212 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260808 11:38:08.461067 1236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:08.462291 1212 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 11:38:08.462459 1212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "5a728eff91e74c46b6128995076f1992"
format_stamp: "Formatted at 2026-08-08 11:38:08 on dist-test-slave-tc2s"
server_key: "a68421a0916298b81e39d7fd0eeba449"
server_key_iv: "3b8ae9fe97eca8d9a52364dc4eaa0829"
server_key_version: "encryptionkey@0"
I20260808 11:38:08.462649 1212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:08.476567 1212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:08.477276 1212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:08.477536 1212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:08.494441 1212 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:32925
I20260808 11:38:08.494459 1320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:32925 every 8 connection(s)
I20260808 11:38:08.495690 1212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:38:08.497833 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 1212
I20260808 11:38:08.498041 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:38:08.498685 24881 external_mini_cluster.cc:1428] Setting key 8cae0b8abb48b2923413fdd724c18e63
I20260808 11:38:08.499253 1321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:08.513494 1324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:08.500419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55452 (local address 127.24.76.126:32925)
0808 11:38:08.501442 (+ 1023us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:08.501456 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:08.501508 (+ 52us) server_negotiation.cc:407] Connection header received
0808 11:38:08.503806 (+ 2298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:08.503832 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:08.504389 (+ 557us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:08.504811 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:08.505340 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:08.507300 (+ 1960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:08.508479 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:08.509459 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:08.509656 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:08.510751 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:08.510761 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:08.510777 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:38:08.511470 (+ 693us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:08.511571 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:38:08.511597 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:08.511705 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:08.511989 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 11:38:08.512293 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":308,"threads_started":1}
I20260808 11:38:08.513574 1321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a728eff91e74c46b6128995076f1992: Bootstrap starting.
I20260808 11:38:08.516640 1321 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a728eff91e74c46b6128995076f1992: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:08.517241 24881 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:38:08.517675 1321 log.cc:824] T 00000000000000000000000000000000 P 5a728eff91e74c46b6128995076f1992: Log is configured to *not* fsync() on all Append() calls
I20260808 11:38:08.520396 1321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a728eff91e74c46b6128995076f1992: No bootstrap required, opened a new log
I20260808 11:38:08.528120 1321 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a728eff91e74c46b6128995076f1992 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a728eff91e74c46b6128995076f1992" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 32925 } }
I20260808 11:38:08.528441 1321 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a728eff91e74c46b6128995076f1992 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:08.528533 1321 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a728eff91e74c46b6128995076f1992 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a728eff91e74c46b6128995076f1992, State: Initialized, Role: FOLLOWER
I20260808 11:38:08.529042 1321 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a728eff91e74c46b6128995076f1992 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a728eff91e74c46b6128995076f1992" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 32925 } }
I20260808 11:38:08.529266 1321 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a728eff91e74c46b6128995076f1992 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:38:08.529385 1321 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a728eff91e74c46b6128995076f1992 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:38:08.529538 1321 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a728eff91e74c46b6128995076f1992 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:08.531737 1321 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a728eff91e74c46b6128995076f1992 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a728eff91e74c46b6128995076f1992" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 32925 } }
I20260808 11:38:08.532226 1321 leader_election.cc:302] T 00000000000000000000000000000000 P 5a728eff91e74c46b6128995076f1992 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a728eff91e74c46b6128995076f1992; no voters:
I20260808 11:38:08.532713 1321 leader_election.cc:288] T 00000000000000000000000000000000 P 5a728eff91e74c46b6128995076f1992 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:38:08.532833 1327 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a728eff91e74c46b6128995076f1992 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:38:08.533522 1327 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a728eff91e74c46b6128995076f1992 [term 1 LEADER]: Becoming Leader. State: Replica: 5a728eff91e74c46b6128995076f1992, State: Running, Role: LEADER
I20260808 11:38:08.534078 1327 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a728eff91e74c46b6128995076f1992 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a728eff91e74c46b6128995076f1992" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 32925 } }
I20260808 11:38:08.534693 1321 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a728eff91e74c46b6128995076f1992 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:38:08.536594 1330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a728eff91e74c46b6128995076f1992 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a728eff91e74c46b6128995076f1992. Latest consensus state: current_term: 1 leader_uuid: "5a728eff91e74c46b6128995076f1992" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a728eff91e74c46b6128995076f1992" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 32925 } } }
I20260808 11:38:08.537041 1330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a728eff91e74c46b6128995076f1992 [sys.catalog]: This master's current role is: LEADER
I20260808 11:38:08.536582 1329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a728eff91e74c46b6128995076f1992 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a728eff91e74c46b6128995076f1992" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a728eff91e74c46b6128995076f1992" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 32925 } } }
I20260808 11:38:08.537612 1329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a728eff91e74c46b6128995076f1992 [sys.catalog]: This master's current role is: LEADER
I20260808 11:38:08.538937 1343 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:38:08.543547 1343 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:38:08.552621 1343 catalog_manager.cc:1347] Generated new cluster ID: 35f3f9cc44d649029161e7833bb1374c
I20260808 11:38:08.552711 1343 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:38:08.602289 1343 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:38:08.603533 1343 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:38:08.608400 1343 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a728eff91e74c46b6128995076f1992: Generated new TSK 0
I20260808 11:38:08.609167 1343 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:38:08.743040 1324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:08.631735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55464 (local address 127.24.76.126:32925)
0808 11:38:08.631917 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:08.631923 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:08.632256 (+ 333us) server_negotiation.cc:407] Connection header received
0808 11:38:08.632482 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:08.632486 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:08.632552 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:08.632663 (+ 111us) server_negotiation.cc:226] Negotiated authn=JWT
0808 11:38:08.633148 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:08.633865 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:08.635963 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:08.636641 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:08.636742 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:08.637277 (+ 535us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 11:38:08.742480 (+105203us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 11:38:08.742809 (+ 329us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55464: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":68}
W20260808 11:38:08.743175 1324 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55464: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 11:38:08.743518 1375 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:08.631548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:32925 (local address 127.0.0.1:55464)
0808 11:38:08.632156 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:08.632175 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:08.632321 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:08.632690 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:08.632695 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:08.632709 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0808 11:38:08.633005 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:08.633020 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:08.634032 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:08.634038 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:08.635821 (+ 1783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:08.635837 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:08.636812 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:08.636818 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:08.637015 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:08.637059 (+ 44us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 11:38:08.742777 (+105718us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 11:38:08.743099 (+ 322us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.76.126:32925: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":288,"threads_started":1}
I20260808 11:38:08.745258 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 1212
I20260808 11:38:08.746430 1316 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:08.850039 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 1212
2026-08-08T11:38:08Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (833 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (407 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (406 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:38:09 dist-test-slave-tc2s krb5kdc[1401](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:38:09 dist-test-slave-tc2s krb5kdc[1401](info): set up 2 sockets
Aug 08 11:38:09 dist-test-slave-tc2s krb5kdc[1401](info): commencing operation
krb5kdc: starting...
W20260808 11:38:10.856693 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:38:10 dist-test-slave-tc2s krb5kdc[1401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:38:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:38:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
I20260808 11:38:11.067382 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:39953
--webserver_interface=127.24.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:38041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:39953
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:38:11.385874 1417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:11.386170 1417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:11.386307 1417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:11.393222 1417 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:38:11.393311 1417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:11.393355 1417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:11.393429 1417 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:38:11.393496 1417 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:38:11.403307 1417 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:38041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:39953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:39953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:11.404886 1417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:11.406666 1417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:11.419251 1424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:11.419284 1428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:11.419880 1425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:11.420176 1417 server_base.cc:1034] running on GCE node
I20260808 11:38:11.420882 1417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:11.422189 1417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:11.423388 1417 hybrid_clock.cc:648] HybridClock initialized: now 1786189091423360 us; error 60 us; skew 500 ppm
I20260808 11:38:11.423710 1417 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:38:11 dist-test-slave-tc2s krb5kdc[1401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189091, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:11.426707 1417 init.cc:377] Logged in from keytab as kudu/127.24.76.126@KRBTEST.COM (short username kudu)
I20260808 11:38:11.428038 1417 webserver.cc:460] Webserver started at http://127.24.76.126:42959/ using document root <none> and password file <none>
I20260808 11:38:11.428673 1417 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:11.428790 1417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:11.429085 1417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:11.431985 1417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "a988996f2c984c57a14d75bc491fdd5e"
format_stamp: "Formatted at 2026-08-08 11:38:11 on dist-test-slave-tc2s"
server_key: "d5a70cc636b7706efd95b92ec22e3984"
server_key_iv: "38cd6a356e9261ccd9a62ed6ecc85dfa"
server_key_version: "encryptionkey@0"
I20260808 11:38:11.432727 1417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "a988996f2c984c57a14d75bc491fdd5e"
format_stamp: "Formatted at 2026-08-08 11:38:11 on dist-test-slave-tc2s"
server_key: "d5a70cc636b7706efd95b92ec22e3984"
server_key_iv: "38cd6a356e9261ccd9a62ed6ecc85dfa"
server_key_version: "encryptionkey@0"
I20260808 11:38:11.437570 1417 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 11:38:11.440752 1438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:11.441886 1417 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 11:38:11.442068 1417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "a988996f2c984c57a14d75bc491fdd5e"
format_stamp: "Formatted at 2026-08-08 11:38:11 on dist-test-slave-tc2s"
server_key: "d5a70cc636b7706efd95b92ec22e3984"
server_key_iv: "38cd6a356e9261ccd9a62ed6ecc85dfa"
server_key_version: "encryptionkey@0"
I20260808 11:38:11.442236 1417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:11.461223 1417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:11.464715 1417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:11.464962 1417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:11.480844 1417 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:39953
I20260808 11:38:11.480897 1513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:39953 every 8 connection(s)
I20260808 11:38:11.482091 1417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:38:11.485467 1514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:11.486196 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 1417
I20260808 11:38:11.486387 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:38:11.487008 24881 external_mini_cluster.cc:1428] Setting key ff8d26ec1c9d5a44d7bf9304e80413ae
I20260808 11:38:11.498965 1514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a988996f2c984c57a14d75bc491fdd5e: Bootstrap starting.
Aug 08 11:38:11 dist-test-slave-tc2s krb5kdc[1401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:38:11.501978 1514 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a988996f2c984c57a14d75bc491fdd5e: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:11.503233 1514 log.cc:824] T 00000000000000000000000000000000 P a988996f2c984c57a14d75bc491fdd5e: Log is configured to *not* fsync() on all Append() calls
I20260808 11:38:11.506314 1514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a988996f2c984c57a14d75bc491fdd5e: No bootstrap required, opened a new log
I20260808 11:38:11.509928 1520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:11.488849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33136 (local address 127.24.76.126:39953)
0808 11:38:11.489663 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:11.489682 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:11.489730 (+ 48us) server_negotiation.cc:407] Connection header received
0808 11:38:11.491618 (+ 1888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:11.491645 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:11.492115 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:11.492543 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:11.493792 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:11.495774 (+ 1982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:11.496965 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:11.497970 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:11.498141 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:11.502056 (+ 3915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:11.502068 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:11.502085 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:11.502126 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:11.505719 (+ 3593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:11.506386 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:11.506394 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:11.506401 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:11.506631 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:11.506972 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:11.506977 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:11.506981 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:11.507652 (+ 671us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:11.507961 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:11.508228 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 11:38:11.508629 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":333,"threads_started":1}
I20260808 11:38:11.515130 1514 raft_consensus.cc:348] T 00000000000000000000000000000000 P a988996f2c984c57a14d75bc491fdd5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a988996f2c984c57a14d75bc491fdd5e" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39953 } }
I20260808 11:38:11.515554 1514 raft_consensus.cc:374] T 00000000000000000000000000000000 P a988996f2c984c57a14d75bc491fdd5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:11.515659 1514 raft_consensus.cc:729] T 00000000000000000000000000000000 P a988996f2c984c57a14d75bc491fdd5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a988996f2c984c57a14d75bc491fdd5e, State: Initialized, Role: FOLLOWER
I20260808 11:38:11.516211 1514 consensus_queue.cc:260] T 00000000000000000000000000000000 P a988996f2c984c57a14d75bc491fdd5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a988996f2c984c57a14d75bc491fdd5e" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39953 } }
I20260808 11:38:11.516425 1514 raft_consensus.cc:388] T 00000000000000000000000000000000 P a988996f2c984c57a14d75bc491fdd5e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:38:11.516567 1514 raft_consensus.cc:482] T 00000000000000000000000000000000 P a988996f2c984c57a14d75bc491fdd5e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:38:11.516710 1514 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a988996f2c984c57a14d75bc491fdd5e [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:11.518797 1514 raft_consensus.cc:504] T 00000000000000000000000000000000 P a988996f2c984c57a14d75bc491fdd5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a988996f2c984c57a14d75bc491fdd5e" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39953 } }
I20260808 11:38:11.519253 1514 leader_election.cc:302] T 00000000000000000000000000000000 P a988996f2c984c57a14d75bc491fdd5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a988996f2c984c57a14d75bc491fdd5e; no voters:
I20260808 11:38:11.519878 1514 leader_election.cc:288] T 00000000000000000000000000000000 P a988996f2c984c57a14d75bc491fdd5e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:38:11.520025 1527 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a988996f2c984c57a14d75bc491fdd5e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:38:11.520718 1527 raft_consensus.cc:686] T 00000000000000000000000000000000 P a988996f2c984c57a14d75bc491fdd5e [term 1 LEADER]: Becoming Leader. State: Replica: a988996f2c984c57a14d75bc491fdd5e, State: Running, Role: LEADER
I20260808 11:38:11.521217 1527 consensus_queue.cc:237] T 00000000000000000000000000000000 P a988996f2c984c57a14d75bc491fdd5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a988996f2c984c57a14d75bc491fdd5e" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39953 } }
I20260808 11:38:11.521808 1514 sys_catalog.cc:564] T 00000000000000000000000000000000 P a988996f2c984c57a14d75bc491fdd5e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:38:11.523722 1531 sys_catalog.cc:455] T 00000000000000000000000000000000 P a988996f2c984c57a14d75bc491fdd5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a988996f2c984c57a14d75bc491fdd5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a988996f2c984c57a14d75bc491fdd5e" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39953 } } }
I20260808 11:38:11.524153 1531 sys_catalog.cc:458] T 00000000000000000000000000000000 P a988996f2c984c57a14d75bc491fdd5e [sys.catalog]: This master's current role is: LEADER
I20260808 11:38:11.523746 1533 sys_catalog.cc:455] T 00000000000000000000000000000000 P a988996f2c984c57a14d75bc491fdd5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a988996f2c984c57a14d75bc491fdd5e. Latest consensus state: current_term: 1 leader_uuid: "a988996f2c984c57a14d75bc491fdd5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a988996f2c984c57a14d75bc491fdd5e" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39953 } } }
I20260808 11:38:11.524411 1533 sys_catalog.cc:458] T 00000000000000000000000000000000 P a988996f2c984c57a14d75bc491fdd5e [sys.catalog]: This master's current role is: LEADER
I20260808 11:38:11.525110 1538 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:38:11.529520 1538 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:38:11.537719 1538 catalog_manager.cc:1347] Generated new cluster ID: 5e9c3d4d5daf429c87754cf056bfa2c4
I20260808 11:38:11.537828 1538 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:38:11.557119 1538 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:38:11.558959 1538 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:38:11.569962 1538 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a988996f2c984c57a14d75bc491fdd5e: Generated new TSK 0
I20260808 11:38:11.570969 1538 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab.
Entry for principal kudu/127.24.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab.
I20260808 11:38:11.633096 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.65:0
--local_ip_for_outbound_sockets=127.24.76.65
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:39953
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:38041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:38:11.957938 1557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:11.958190 1557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:11.958321 1557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:11.965240 1557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:11.965337 1557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:11.965498 1557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.65
I20260808 11:38:11.974867 1557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:38041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:39953
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.65
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:11.976037 1557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:11.977247 1557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:11.989506 1567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:11.989583 1568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:11.989604 1570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:11.990442 1557 server_base.cc:1034] running on GCE node
I20260808 11:38:11.990988 1557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:11.991837 1557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:11.993033 1557 hybrid_clock.cc:648] HybridClock initialized: now 1786189091992994 us; error 29 us; skew 500 ppm
I20260808 11:38:11.993312 1557 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:38:11 dist-test-slave-tc2s krb5kdc[1401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189091, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:11.996241 1557 init.cc:377] Logged in from keytab as kudu/127.24.76.65@KRBTEST.COM (short username kudu)
I20260808 11:38:11.997572 1557 webserver.cc:460] Webserver started at http://127.24.76.65:35453/ using document root <none> and password file <none>
I20260808 11:38:11.998243 1557 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:11.998365 1557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:11.998652 1557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:12.001735 1557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-0/data/instance:
uuid: "0a80e6703bef4b939405b81a1664710f"
format_stamp: "Formatted at 2026-08-08 11:38:11 on dist-test-slave-tc2s"
server_key: "0b2b6286b17f5a85ebe90bb1547587ae"
server_key_iv: "da219b7618c8864a32b43e9de18908f2"
server_key_version: "encryptionkey@0"
I20260808 11:38:12.002525 1557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance:
uuid: "0a80e6703bef4b939405b81a1664710f"
format_stamp: "Formatted at 2026-08-08 11:38:11 on dist-test-slave-tc2s"
server_key: "0b2b6286b17f5a85ebe90bb1547587ae"
server_key_iv: "da219b7618c8864a32b43e9de18908f2"
server_key_version: "encryptionkey@0"
I20260808 11:38:12.007505 1557 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 11:38:12.010749 1580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:12.011996 1557 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 11:38:12.012142 1557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-0/wal
uuid: "0a80e6703bef4b939405b81a1664710f"
format_stamp: "Formatted at 2026-08-08 11:38:11 on dist-test-slave-tc2s"
server_key: "0b2b6286b17f5a85ebe90bb1547587ae"
server_key_iv: "da219b7618c8864a32b43e9de18908f2"
server_key_version: "encryptionkey@0"
I20260808 11:38:12.012316 1557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:12.032091 1557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:12.035761 1557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:12.036015 1557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:12.036906 1557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:38:12.038232 1557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:38:12.038312 1557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:12.038432 1557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:38:12.038494 1557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:12.068444 1557 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.65:37093
I20260808 11:38:12.068502 1757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.65:37093 every 8 connection(s)
I20260808 11:38:12.069686 1557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
I20260808 11:38:12.075026 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 1557
I20260808 11:38:12.075234 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance
I20260808 11:38:12.076045 24881 external_mini_cluster.cc:1428] Setting key 210148ac9b5570afc1c3219b7e5fad84
Aug 08 11:38:12 dist-test-slave-tc2s krb5kdc[1401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189091, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:38:12.089881 1761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:12.072113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:51423 (local address 127.24.76.126:39953)
0808 11:38:12.072596 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:12.072600 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:12.073513 (+ 913us) server_negotiation.cc:407] Connection header received
0808 11:38:12.075584 (+ 2071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:12.075589 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:12.075664 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:12.075759 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:12.077419 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:12.078340 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:12.080442 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:12.080840 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:12.080927 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:12.084536 (+ 3609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:12.084540 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:12.084544 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:12.084575 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:12.087066 (+ 2491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:12.087747 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:12.087751 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:12.087753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:12.087868 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:12.088271 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:12.088276 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:12.088278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:12.088523 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:12.088655 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:12.089313 (+ 658us) server_negotiation.cc:299] Negotiation successful
0808 11:38:12.089538 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":236,"threads_started":1}
I20260808 11:38:12.090776 1762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:12.072687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:39953 (local address 127.24.76.65:51423)
0808 11:38:12.073364 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:12.073402 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:12.075292 (+ 1890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:12.076021 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:12.076033 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:12.076736 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:12.077261 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:12.077278 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:12.078511 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:12.078516 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:12.080319 (+ 1803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:12.080335 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:12.081046 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:12.081052 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:12.081314 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:12.081972 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:12.082006 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:12.084338 (+ 2332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:12.087195 (+ 2857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:12.087203 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:12.087218 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:12.087634 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:12.087971 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:12.087976 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:12.087980 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:12.088165 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:12.088645 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:12.088652 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:12.088960 (+ 308us) client_negotiation.cc:769] Sending connection context
0808 11:38:12.089194 (+ 234us) client_negotiation.cc:240] Negotiation successful
0808 11:38:12.089682 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":259,"threads_started":1}
I20260808 11:38:12.092325 1758 heartbeater.cc:347] Connected to a master server at 127.24.76.126:39953
I20260808 11:38:12.092598 1758 heartbeater.cc:464] Registering TS with master...
I20260808 11:38:12.093266 1758 heartbeater.cc:511] Master 127.24.76.126:39953 requested a full tablet report, sending...
I20260808 11:38:12.095275 1462 ts_manager.cc:194] Registered new tserver with Master: 0a80e6703bef4b939405b81a1664710f (127.24.76.65:37093)
I20260808 11:38:12.096668 1462 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.65@KRBTEST.COM'} at 127.24.76.65:51423
WARNING: no policy specified for kudu/127.24.76.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab.
Entry for principal kudu/127.24.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab.
I20260808 11:38:12.126595 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.66:0
--local_ip_for_outbound_sockets=127.24.76.66
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:39953
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:38041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:38:12.444576 1765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:12.444832 1765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:12.444936 1765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:12.452036 1765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:12.452135 1765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:12.452250 1765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.66
I20260808 11:38:12.461387 1765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:38041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:39953
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.66
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:12.462561 1765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:12.463755 1765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:12.476284 1772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:12.476318 1776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:12.476244 1778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:12.477030 1765 server_base.cc:1034] running on GCE node
I20260808 11:38:12.477581 1765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:12.478448 1765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:12.479643 1765 hybrid_clock.cc:648] HybridClock initialized: now 1786189092479597 us; error 42 us; skew 500 ppm
I20260808 11:38:12.479919 1765 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:38:12 dist-test-slave-tc2s krb5kdc[1401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189092, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:12.482892 1765 init.cc:377] Logged in from keytab as kudu/127.24.76.66@KRBTEST.COM (short username kudu)
I20260808 11:38:12.484373 1765 webserver.cc:460] Webserver started at http://127.24.76.66:46255/ using document root <none> and password file <none>
I20260808 11:38:12.484998 1765 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:12.485106 1765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:12.485391 1765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:12.488152 1765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-1/data/instance:
uuid: "8ab945d5d5894ce6b54e75c9975b5c14"
format_stamp: "Formatted at 2026-08-08 11:38:12 on dist-test-slave-tc2s"
server_key: "2d3c040a426fe55b0d8044e3a9a869ec"
server_key_iv: "367531b6f59248a75a7afd97860d3232"
server_key_version: "encryptionkey@0"
I20260808 11:38:12.488875 1765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance:
uuid: "8ab945d5d5894ce6b54e75c9975b5c14"
format_stamp: "Formatted at 2026-08-08 11:38:12 on dist-test-slave-tc2s"
server_key: "2d3c040a426fe55b0d8044e3a9a869ec"
server_key_iv: "367531b6f59248a75a7afd97860d3232"
server_key_version: "encryptionkey@0"
I20260808 11:38:12.493811 1765 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.007s
I20260808 11:38:12.497046 1789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:12.498281 1765 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 11:38:12.498409 1765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-1/wal
uuid: "8ab945d5d5894ce6b54e75c9975b5c14"
format_stamp: "Formatted at 2026-08-08 11:38:12 on dist-test-slave-tc2s"
server_key: "2d3c040a426fe55b0d8044e3a9a869ec"
server_key_iv: "367531b6f59248a75a7afd97860d3232"
server_key_version: "encryptionkey@0"
I20260808 11:38:12.498579 1765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:12.525224 1765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:12.529237 1765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:12.529467 1765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:12.530373 1765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:38:12.531603 1765 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:38:12.531674 1765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:12.531760 1765 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:38:12.531803 1765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:12.568893 1765 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.66:39641
I20260808 11:38:12.568965 1958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.66:39641 every 8 connection(s)
I20260808 11:38:12.570467 1765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
I20260808 11:38:12.578157 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 1765
I20260808 11:38:12.578315 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance
I20260808 11:38:12.578893 24881 external_mini_cluster.cc:1428] Setting key 07162e206845cf7127aa6ec9838243c6
Aug 08 11:38:12 dist-test-slave-tc2s krb5kdc[1401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189092, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:38:12.590523 1761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:12.572887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:44599 (local address 127.24.76.126:39953)
0808 11:38:12.573101 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:12.573107 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:12.574279 (+ 1172us) server_negotiation.cc:407] Connection header received
0808 11:38:12.576015 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:12.576019 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:12.576089 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:12.576176 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:12.577534 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:12.578353 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:12.580340 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:12.580987 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:12.581110 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:12.584895 (+ 3785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:12.584901 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:12.584905 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:12.584941 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:12.587768 (+ 2827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:12.588436 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:12.588441 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:12.588444 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:12.588601 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:12.588960 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:12.588964 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:12.588966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:12.589261 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:12.589398 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:12.589991 (+ 593us) server_negotiation.cc:299] Negotiation successful
0808 11:38:12.590199 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 11:38:12.591439 1965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:12.573397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:39953 (local address 127.24.76.66:44599)
0808 11:38:12.574124 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:12.574166 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:12.575777 (+ 1611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:12.576351 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:12.576358 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:12.576849 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:12.577380 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:12.577397 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:12.578511 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:12.578516 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:12.580175 (+ 1659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:12.580192 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:12.581221 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:12.581228 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:12.581511 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:12.582204 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:12.582229 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:12.584620 (+ 2391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:12.587893 (+ 3273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:12.587899 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:12.587911 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:12.588329 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:12.588689 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:12.588693 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:12.588695 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:12.588868 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:12.589359 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:12.589365 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:12.589648 (+ 283us) client_negotiation.cc:769] Sending connection context
0808 11:38:12.589863 (+ 215us) client_negotiation.cc:240] Negotiation successful
0808 11:38:12.590279 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":286,"threads_started":1}
I20260808 11:38:12.592770 1960 heartbeater.cc:347] Connected to a master server at 127.24.76.126:39953
I20260808 11:38:12.593056 1960 heartbeater.cc:464] Registering TS with master...
I20260808 11:38:12.593698 1960 heartbeater.cc:511] Master 127.24.76.126:39953 requested a full tablet report, sending...
I20260808 11:38:12.595187 1462 ts_manager.cc:194] Registered new tserver with Master: 8ab945d5d5894ce6b54e75c9975b5c14 (127.24.76.66:39641)
I20260808 11:38:12.596274 1462 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.66@KRBTEST.COM'} at 127.24.76.66:44599
WARNING: no policy specified for kudu/127.24.76.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab.
Entry for principal kudu/127.24.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab.
I20260808 11:38:12.631232 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.67:0
--local_ip_for_outbound_sockets=127.24.76.67
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:39953
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:38041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:38:12.948186 1971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:12.948444 1971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:12.948580 1971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:12.955494 1971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:12.955591 1971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:12.955739 1971 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.67
I20260808 11:38:12.964893 1971 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:38041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:39953
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.67
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:12.966070 1971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:12.967305 1971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:12.979560 1982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:12.979663 1978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:12.979667 1988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:12.980263 1971 server_base.cc:1034] running on GCE node
I20260808 11:38:12.980819 1971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:12.981669 1971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:12.982860 1971 hybrid_clock.cc:648] HybridClock initialized: now 1786189092982798 us; error 49 us; skew 500 ppm
I20260808 11:38:12.983150 1971 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:38:12 dist-test-slave-tc2s krb5kdc[1401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189092, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:12.986131 1971 init.cc:377] Logged in from keytab as kudu/127.24.76.67@KRBTEST.COM (short username kudu)
I20260808 11:38:12.987401 1971 webserver.cc:460] Webserver started at http://127.24.76.67:46517/ using document root <none> and password file <none>
I20260808 11:38:12.988051 1971 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:12.988163 1971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:12.988445 1971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:12.991299 1971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-2/data/instance:
uuid: "9652f1f535ab4e77985fafaf0cf510ae"
format_stamp: "Formatted at 2026-08-08 11:38:12 on dist-test-slave-tc2s"
server_key: "fca39b609400943710afd7ab0e0bbe89"
server_key_iv: "18dcdaa9e5d3c58b011f5b66f77ba69d"
server_key_version: "encryptionkey@0"
I20260808 11:38:12.992071 1971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance:
uuid: "9652f1f535ab4e77985fafaf0cf510ae"
format_stamp: "Formatted at 2026-08-08 11:38:12 on dist-test-slave-tc2s"
server_key: "fca39b609400943710afd7ab0e0bbe89"
server_key_iv: "18dcdaa9e5d3c58b011f5b66f77ba69d"
server_key_version: "encryptionkey@0"
I20260808 11:38:12.996820 1971 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 11:38:13.000049 2002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:13.001305 1971 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 11:38:13.001446 1971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-2/wal
uuid: "9652f1f535ab4e77985fafaf0cf510ae"
format_stamp: "Formatted at 2026-08-08 11:38:12 on dist-test-slave-tc2s"
server_key: "fca39b609400943710afd7ab0e0bbe89"
server_key_iv: "18dcdaa9e5d3c58b011f5b66f77ba69d"
server_key_version: "encryptionkey@0"
I20260808 11:38:13.001619 1971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:13.016227 1971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:13.020143 1971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:13.020401 1971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:13.021283 1971 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:38:13.022513 1971 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:38:13.022584 1971 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:13.022696 1971 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:38:13.022754 1971 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:13.052181 1971 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.67:37255
I20260808 11:38:13.052222 2181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.67:37255 every 8 connection(s)
I20260808 11:38:13.053378 1971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
I20260808 11:38:13.059377 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 1971
I20260808 11:38:13.059556 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance
I20260808 11:38:13.060104 24881 external_mini_cluster.cc:1428] Setting key d689b14abe2abe1d3a85fd81242194a3
Aug 08 11:38:13 dist-test-slave-tc2s krb5kdc[1401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189092, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:38:13.072839 1761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:13.055705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:33873 (local address 127.24.76.126:39953)
0808 11:38:13.055878 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:13.055883 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:13.057008 (+ 1125us) server_negotiation.cc:407] Connection header received
0808 11:38:13.058763 (+ 1755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:13.058768 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:13.058837 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:13.058914 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:13.060835 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:13.061644 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:13.063164 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:13.063558 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:13.063666 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:13.066834 (+ 3168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:13.066839 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:13.066842 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:13.066873 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:13.069846 (+ 2973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:13.070536 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:13.070542 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:13.070546 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:13.070701 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:13.071087 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:13.071092 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:13.071096 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:13.071429 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:13.071574 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:13.072279 (+ 705us) server_negotiation.cc:299] Negotiation successful
0808 11:38:13.072493 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 11:38:13.073587 2186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:13.056157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:39953 (local address 127.24.76.67:33873)
0808 11:38:13.056844 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:13.056883 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:13.058517 (+ 1634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:13.059172 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:13.059185 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:13.059893 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:13.060670 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:13.060691 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:13.061800 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:13.061804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:13.063026 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:13.063039 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:13.063773 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:13.063780 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:13.063999 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:13.064546 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:13.064567 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:13.066656 (+ 2089us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:13.070017 (+ 3361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:13.070024 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:13.070034 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:13.070414 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:13.070794 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:13.070797 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:13.070799 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:13.070969 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:13.071558 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:13.071565 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:13.071940 (+ 375us) client_negotiation.cc:769] Sending connection context
0808 11:38:13.072152 (+ 212us) client_negotiation.cc:240] Negotiation successful
0808 11:38:13.072504 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":276,"threads_started":1}
I20260808 11:38:13.074887 2184 heartbeater.cc:347] Connected to a master server at 127.24.76.126:39953
I20260808 11:38:13.075209 2184 heartbeater.cc:464] Registering TS with master...
I20260808 11:38:13.075896 2184 heartbeater.cc:511] Master 127.24.76.126:39953 requested a full tablet report, sending...
I20260808 11:38:13.077116 1462 ts_manager.cc:194] Registered new tserver with Master: 9652f1f535ab4e77985fafaf0cf510ae (127.24.76.67:37255)
I20260808 11:38:13.077885 1462 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.67@KRBTEST.COM'} at 127.24.76.67:33873
I20260808 11:38:13.087152 24881 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:38:13.099365 1758 heartbeater.cc:503] Master 127.24.76.126:39953 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/krb5kdc/test-admin.keytab.
Aug 08 11:38:13 dist-test-slave-tc2s krb5kdc[1401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:13.117066 24881 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 11:38:13.121980 2195 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:13.119513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35250 (local address 127.24.76.65:37093)
0808 11:38:13.120163 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:13.120170 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:13.120194 (+ 24us) server_negotiation.cc:407] Connection header received
0808 11:38:13.120247 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:13.120253 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:13.120467 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:13.120622 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:13.121523 (+ 901us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35250: BlockingRecv error: recv got EOF from 127.0.0.1:35250 (error 108)
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":317,"threads_started":1}
W20260808 11:38:13.121976 24881 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/krb5kdc/krb5cc' of type FILE
Aug 08 11:38:13 dist-test-slave-tc2s krb5kdc[1401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:13.123615 24881 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 11:38:13 dist-test-slave-tc2s krb5kdc[1401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:13.124881 24881 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 11:38:13 dist-test-slave-tc2s krb5kdc[1401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.65@KRBTEST.COM
I20260808 11:38:13.139020 2195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:13.126333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35252 (local address 127.24.76.65:37093)
0808 11:38:13.126491 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:13.126497 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:13.126747 (+ 250us) server_negotiation.cc:407] Connection header received
0808 11:38:13.126900 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:13.126905 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:13.126989 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:13.127069 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:13.128072 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:13.129004 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:13.130264 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:13.130796 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:13.130906 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:13.134199 (+ 3293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:13.134210 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:13.134223 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:13.134253 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:13.136745 (+ 2492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:13.137330 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:13.137336 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:13.137338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:13.137454 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:13.137715 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:13.137718 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:13.137720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:13.138095 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:13.138316 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:13.138566 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 11:38:13.138750 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260808 11:38:13.144073 2195 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:13.142604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35254 (local address 127.24.76.65:37093)
0808 11:38:13.142789 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:13.142794 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:13.142955 (+ 161us) server_negotiation.cc:407] Connection header received
0808 11:38:13.143072 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:13.143076 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:13.143128 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:13.143196 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:13.143848 (+ 652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35254: BlockingRecv error: recv got EOF from 127.0.0.1:35254 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260808 11:38:13.144230 24881 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/krb5kdc/krb5cc' of type FILE
Aug 08 11:38:13 dist-test-slave-tc2s krb5kdc[1401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:13.145550 24881 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 11:38:13 dist-test-slave-tc2s krb5kdc[1401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:13.146767 24881 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 11:38:13 dist-test-slave-tc2s krb5kdc[1401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.65@KRBTEST.COM
I20260808 11:38:13.159312 2195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:13.147851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35258 (local address 127.24.76.65:37093)
0808 11:38:13.148015 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:13.148021 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:13.148271 (+ 250us) server_negotiation.cc:407] Connection header received
0808 11:38:13.148406 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:13.148411 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:13.148466 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:13.148540 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:13.149431 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:13.150160 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:13.151279 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:13.151702 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:13.151805 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:13.154875 (+ 3070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:13.154880 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:13.154883 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:13.154912 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:13.157049 (+ 2137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:13.157610 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:13.157614 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:13.157616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:13.157731 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:13.158129 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:13.158134 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:13.158138 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:13.158381 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:13.158622 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:13.158899 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 11:38:13.159093 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260808 11:38:13.162745 2195 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:13.161359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35270 (local address 127.24.76.65:37093)
0808 11:38:13.161543 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:13.161548 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:13.161728 (+ 180us) server_negotiation.cc:407] Connection header received
0808 11:38:13.161841 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:13.161844 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:13.161934 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:13.162003 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:13.162576 (+ 573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35270: BlockingRecv error: recv got EOF from 127.0.0.1:35270 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260808 11:38:13.162918 24881 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786189042529715-24881-0/krb5kdc/krb5cc' of type FILE
Aug 08 11:38:13 dist-test-slave-tc2s krb5kdc[1401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:13.164237 24881 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 11:38:13 dist-test-slave-tc2s krb5kdc[1401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:13.165343 24881 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 11:38:13 dist-test-slave-tc2s krb5kdc[1401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.65@KRBTEST.COM
I20260808 11:38:13.178122 2195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:13.166466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35280 (local address 127.24.76.65:37093)
0808 11:38:13.166618 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:13.166623 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:13.166891 (+ 268us) server_negotiation.cc:407] Connection header received
0808 11:38:13.167029 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:13.167033 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:13.167081 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:13.167150 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:13.168015 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:13.168691 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:13.169872 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:13.170298 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:13.170403 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:13.173443 (+ 3040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:13.173456 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:13.173459 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:13.173486 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:13.175796 (+ 2310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:13.176372 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:13.176375 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:13.176377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:13.176530 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:13.176871 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:13.176875 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:13.176877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:13.177121 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:13.177402 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:13.177646 (+ 244us) server_negotiation.cc:299] Negotiation successful
0808 11:38:13.177838 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 11:38:13.179352 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 1557
I20260808 11:38:13.190133 2195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:13.179938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35294 (local address 127.24.76.65:37093)
0808 11:38:13.180089 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:13.180095 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:13.180371 (+ 276us) server_negotiation.cc:407] Connection header received
0808 11:38:13.180516 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:13.180520 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:13.180569 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:13.180639 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:13.181518 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:13.182202 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:13.183333 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:13.183734 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:13.183848 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:13.185602 (+ 1754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:13.185607 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:13.185610 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:13.185636 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:13.187709 (+ 2073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:13.188356 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:13.188362 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:13.188364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:13.188505 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:13.188871 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:13.188875 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:13.188877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:13.189110 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:13.189363 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:13.189647 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 11:38:13.189835 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 11:38:13.190658 1752 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:13.273030 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 1557
I20260808 11:38:13.298043 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 1765
Aug 08 11:38:13 dist-test-slave-tc2s krb5kdc[1401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.66@KRBTEST.COM
I20260808 11:38:13.313323 2213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:13.298917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35674 (local address 127.24.76.66:39641)
0808 11:38:13.299489 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:13.299496 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:13.299524 (+ 28us) server_negotiation.cc:407] Connection header received
0808 11:38:13.299587 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:13.299593 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:13.299797 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:13.299925 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:13.300892 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:13.302112 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:13.303361 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:13.303903 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:13.304013 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:13.307772 (+ 3759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:13.307783 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:13.307798 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:13.307830 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:13.310710 (+ 2880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:13.311365 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:13.311372 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:13.311374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:13.311499 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:13.311838 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:13.311842 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:13.311844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:13.312172 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:13.312488 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:13.312787 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 11:38:13.313030 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":265,"threads_started":1}
I20260808 11:38:13.314563 1950 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:13.395463 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 1765
I20260808 11:38:13.422189 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 1971
Aug 08 11:38:13 dist-test-slave-tc2s krb5kdc[1401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.67@KRBTEST.COM
I20260808 11:38:13.437363 2216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:13.423280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40794 (local address 127.24.76.67:37255)
0808 11:38:13.423965 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:13.423972 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:13.424002 (+ 30us) server_negotiation.cc:407] Connection header received
0808 11:38:13.424064 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:13.424071 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:13.424284 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:13.424424 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:13.425420 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:13.426635 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:13.427850 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:13.428447 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:13.428541 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:13.432218 (+ 3677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:13.432229 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:13.432244 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:13.432275 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:13.435061 (+ 2786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:13.435643 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:13.435649 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:13.435652 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:13.435767 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:13.436043 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:13.436046 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:13.436048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:13.436368 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:13.436589 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:13.436794 (+ 205us) server_negotiation.cc:299] Negotiation successful
0808 11:38:13.437021 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":375,"threads_started":1}
I20260808 11:38:13.438680 2174 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:13.519577 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 1971
I20260808 11:38:13.540925 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 1417
I20260808 11:38:13.542153 1507 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:13.643584 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 1417
2026-08-08T11:38:13Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3985 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:38:13 dist-test-slave-tc2s krb5kdc[2224](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:38:13 dist-test-slave-tc2s krb5kdc[2224](info): set up 2 sockets
Aug 08 11:38:13 dist-test-slave-tc2s krb5kdc[2224](info): commencing operation
krb5kdc: starting...
W20260808 11:38:14.848892 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.006s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:38:14 dist-test-slave-tc2s krb5kdc[2224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:38:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:38:15Z Disabled control of system clock
WARNING: no policy specified for oryx/127.24.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.126.keytab.
Entry for principal oryx/127.24.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.126.keytab.
I20260808 11:38:15.069042 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:44823
--webserver_interface=127.24.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.126.keytab
--principal=oryx/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:44797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:44823
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:38:15.388183 2241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:15.388477 2241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:15.388613 2241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:15.395529 2241 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:38:15.395637 2241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:15.395715 2241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:15.395790 2241 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:38:15.395851 2241 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:38:15.406744 2241 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:44797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:44823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.126.keytab
--principal=oryx/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:44823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:15.408375 2241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:15.410149 2241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:15.421546 2254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:15.421651 2250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:15.421623 2248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:15.422545 2241 server_base.cc:1034] running on GCE node
I20260808 11:38:15.423273 2241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:15.424569 2241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:15.425769 2241 hybrid_clock.cc:648] HybridClock initialized: now 1786189095425714 us; error 41 us; skew 500 ppm
I20260808 11:38:15.426117 2241 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:38:15 dist-test-slave-tc2s krb5kdc[2224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189095, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:15.431710 2241 init.cc:377] Logged in from keytab as oryx/127.24.76.126@KRBTEST.COM (short username oryx)
I20260808 11:38:15.433110 2241 webserver.cc:460] Webserver started at http://127.24.76.126:34227/ using document root <none> and password file <none>
I20260808 11:38:15.433741 2241 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:15.433854 2241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:15.434163 2241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:15.436988 2241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "6ca9db23e8f945df88bffa6f2836ea9a"
format_stamp: "Formatted at 2026-08-08 11:38:15 on dist-test-slave-tc2s"
server_key: "d01a0d589a47e313ba2acf1af35998e4"
server_key_iv: "ed96ebd2b4c94451c7e2261c0640cfaa"
server_key_version: "encryptionkey@0"
I20260808 11:38:15.437716 2241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "6ca9db23e8f945df88bffa6f2836ea9a"
format_stamp: "Formatted at 2026-08-08 11:38:15 on dist-test-slave-tc2s"
server_key: "d01a0d589a47e313ba2acf1af35998e4"
server_key_iv: "ed96ebd2b4c94451c7e2261c0640cfaa"
server_key_version: "encryptionkey@0"
I20260808 11:38:15.442509 2241 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 11:38:15.445657 2262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:15.446911 2241 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 11:38:15.447050 2241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "6ca9db23e8f945df88bffa6f2836ea9a"
format_stamp: "Formatted at 2026-08-08 11:38:15 on dist-test-slave-tc2s"
server_key: "d01a0d589a47e313ba2acf1af35998e4"
server_key_iv: "ed96ebd2b4c94451c7e2261c0640cfaa"
server_key_version: "encryptionkey@0"
I20260808 11:38:15.447204 2241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:15.477102 2241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:15.481132 2241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:15.481354 2241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:15.498109 2241 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:44823
I20260808 11:38:15.498140 2345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:44823 every 8 connection(s)
I20260808 11:38:15.499377 2241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:38:15.502849 2347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:15.508762 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 2241
I20260808 11:38:15.508944 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:38:15.509594 24881 external_mini_cluster.cc:1428] Setting key fa302772b06dc9399000e530d973b2ce
I20260808 11:38:15.512982 2347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a: Bootstrap starting.
I20260808 11:38:15.516407 2347 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:15.517419 2347 log.cc:824] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a: Log is configured to *not* fsync() on all Append() calls
I20260808 11:38:15.519981 2347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a: No bootstrap required, opened a new log
Aug 08 11:38:15 dist-test-slave-tc2s krb5kdc[2224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM
I20260808 11:38:15.527168 2347 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } }
I20260808 11:38:15.527522 2347 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:15.527609 2347 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ca9db23e8f945df88bffa6f2836ea9a, State: Initialized, Role: FOLLOWER
I20260808 11:38:15.528184 2347 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } }
I20260808 11:38:15.528419 2347 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:38:15.528578 2347 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:38:15.528748 2347 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:15.530429 2351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:15.511465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36950 (local address 127.24.76.126:44823)
0808 11:38:15.512180 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:15.512191 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:15.512232 (+ 41us) server_negotiation.cc:407] Connection header received
0808 11:38:15.513851 (+ 1619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:15.513875 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:15.514268 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:15.514569 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:15.516410 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:15.517733 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:15.518961 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:15.519619 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:15.519777 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:15.523461 (+ 3684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:15.523468 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:15.523485 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:15.523529 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:15.526527 (+ 2998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:15.527143 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:15.527149 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:15.527159 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:15.527322 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:15.527705 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:15.527710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:15.527714 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:15.528249 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:15.528540 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:15.528977 (+ 437us) server_negotiation.cc:299] Negotiation successful
0808 11:38:15.529383 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":313,"threads_started":1}
I20260808 11:38:15.531224 2347 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } }
I20260808 11:38:15.531723 2347 leader_election.cc:302] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ca9db23e8f945df88bffa6f2836ea9a; no voters:
I20260808 11:38:15.532279 2347 leader_election.cc:288] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:38:15.532419 2355 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:38:15.533674 2355 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [term 1 LEADER]: Becoming Leader. State: Replica: 6ca9db23e8f945df88bffa6f2836ea9a, State: Running, Role: LEADER
I20260808 11:38:15.534198 2347 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:38:15.534161 2355 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } }
I20260808 11:38:15.536597 2357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ca9db23e8f945df88bffa6f2836ea9a. Latest consensus state: current_term: 1 leader_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } }
I20260808 11:38:15.536947 2357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [sys.catalog]: This master's current role is: LEADER
I20260808 11:38:15.537477 2356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } }
I20260808 11:38:15.537827 2356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [sys.catalog]: This master's current role is: LEADER
W20260808 11:38:15.550459 2378 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 11:38:15.550590 2378 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 11:38:15.550720 2363 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:38:15.552419 2363 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:38:15.560334 2363 catalog_manager.cc:1347] Generated new cluster ID: 76425cb76ac444de9d8e156385ebe8f6
I20260808 11:38:15.560442 2363 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:38:15.577478 2363 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:38:15.579232 2363 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:38:15.600266 2363 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a: Generated new TSK 0
I20260808 11:38:15.601286 2363 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.24.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.65.keytab.
Entry for principal oryx/127.24.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.65.keytab.
I20260808 11:38:15.671165 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.65:0
--local_ip_for_outbound_sockets=127.24.76.65
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:44823
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.65.keytab
--principal=oryx/127.24.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:44797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:38:15.995359 2390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:15.995607 2390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:15.995738 2390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:16.002614 2390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:16.002714 2390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:16.002872 2390 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.65
I20260808 11:38:16.011966 2390 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:44797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.65.keytab
--principal=oryx/127.24.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:44823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.65
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:16.013111 2390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:16.014348 2390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:16.026768 2397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:16.026712 2400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:16.026687 2398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:16.027560 2390 server_base.cc:1034] running on GCE node
I20260808 11:38:16.028118 2390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:16.028945 2390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:16.030138 2390 hybrid_clock.cc:648] HybridClock initialized: now 1786189096030093 us; error 33 us; skew 500 ppm
I20260808 11:38:16.030408 2390 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:38:16 dist-test-slave-tc2s krb5kdc[2224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189096, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:16.033413 2390 init.cc:377] Logged in from keytab as oryx/127.24.76.65@KRBTEST.COM (short username oryx)
I20260808 11:38:16.034765 2390 webserver.cc:460] Webserver started at http://127.24.76.65:46385/ using document root <none> and password file <none>
I20260808 11:38:16.035377 2390 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:16.035497 2390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:16.035777 2390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:16.038573 2390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-0/data/instance:
uuid: "900623cd4f4a40c99098e6dfbe9f684f"
format_stamp: "Formatted at 2026-08-08 11:38:16 on dist-test-slave-tc2s"
server_key: "12eea97d396a69a65e3afdc857947489"
server_key_iv: "f6790a7f5f73f0a5d0a3e2160f0e541f"
server_key_version: "encryptionkey@0"
I20260808 11:38:16.039300 2390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance:
uuid: "900623cd4f4a40c99098e6dfbe9f684f"
format_stamp: "Formatted at 2026-08-08 11:38:16 on dist-test-slave-tc2s"
server_key: "12eea97d396a69a65e3afdc857947489"
server_key_iv: "f6790a7f5f73f0a5d0a3e2160f0e541f"
server_key_version: "encryptionkey@0"
I20260808 11:38:16.044291 2390 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260808 11:38:16.047581 2408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:16.048844 2390 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 11:38:16.048990 2390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-0/wal
uuid: "900623cd4f4a40c99098e6dfbe9f684f"
format_stamp: "Formatted at 2026-08-08 11:38:16 on dist-test-slave-tc2s"
server_key: "12eea97d396a69a65e3afdc857947489"
server_key_iv: "f6790a7f5f73f0a5d0a3e2160f0e541f"
server_key_version: "encryptionkey@0"
I20260808 11:38:16.049165 2390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:16.066905 2390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:16.070381 2390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:16.070627 2390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:16.071484 2390 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:38:16.072726 2390 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:38:16.072795 2390 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:16.072916 2390 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:38:16.072971 2390 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:16.102799 2390 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.65:33613
I20260808 11:38:16.102828 2570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.65:33613 every 8 connection(s)
I20260808 11:38:16.104044 2390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
I20260808 11:38:16.112573 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 2390
I20260808 11:38:16.112785 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance
I20260808 11:38:16.113607 24881 external_mini_cluster.cc:1428] Setting key 38c483571340438c7410d7e27dbe5ea3
Aug 08 11:38:16 dist-test-slave-tc2s krb5kdc[2224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189096, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM
I20260808 11:38:16.123929 2575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:16.106405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:45399 (local address 127.24.76.126:44823)
0808 11:38:16.106845 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:16.106849 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:16.107678 (+ 829us) server_negotiation.cc:407] Connection header received
0808 11:38:16.109427 (+ 1749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:16.109432 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:16.109512 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:16.109608 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:16.111074 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:16.111784 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:16.114000 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:16.114401 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:16.114532 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:16.118203 (+ 3671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:16.118209 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:16.118213 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:16.118248 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:16.121080 (+ 2832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:16.121735 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:16.121741 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:16.121745 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:16.121885 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:16.122302 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:16.122307 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:16.122310 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:16.122626 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:16.122762 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:16.123401 (+ 639us) server_negotiation.cc:299] Negotiation successful
0808 11:38:16.123626 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":228,"threads_started":1}
I20260808 11:38:16.124624 2576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:16.106838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.24.76.65:45399)
0808 11:38:16.107526 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:16.107562 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:16.109167 (+ 1605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:16.109847 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:16.109855 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:16.110373 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:16.110907 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:16.110924 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:16.111932 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:16.111936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:16.113819 (+ 1883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:16.113834 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:16.114628 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:16.114633 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:16.114888 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:16.115548 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:16.115575 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:16.118001 (+ 2426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:16.121203 (+ 3202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:16.121209 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:16.121220 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:16.121616 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:16.122014 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:16.122017 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:16.122019 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:16.122196 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:16.122722 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:16.122728 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:16.123014 (+ 286us) client_negotiation.cc:769] Sending connection context
0808 11:38:16.123244 (+ 230us) client_negotiation.cc:240] Negotiation successful
0808 11:38:16.123604 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":249,"threads_started":1}
I20260808 11:38:16.126101 2572 heartbeater.cc:347] Connected to a master server at 127.24.76.126:44823
I20260808 11:38:16.126410 2572 heartbeater.cc:464] Registering TS with master...
I20260808 11:38:16.127236 2572 heartbeater.cc:511] Master 127.24.76.126:44823 requested a full tablet report, sending...
I20260808 11:38:16.129223 2294 ts_manager.cc:194] Registered new tserver with Master: 900623cd4f4a40c99098e6dfbe9f684f (127.24.76.65:33613)
I20260808 11:38:16.130627 2294 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.76.65@KRBTEST.COM'} at 127.24.76.65:45399
WARNING: no policy specified for oryx/127.24.76.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.76.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.66.keytab.
Entry for principal oryx/127.24.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.66.keytab.
I20260808 11:38:16.164978 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.66:0
--local_ip_for_outbound_sockets=127.24.76.66
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:44823
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.66.keytab
--principal=oryx/127.24.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:44797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:38:16.484849 2579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:16.485123 2579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:16.485275 2579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:16.492275 2579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:16.492396 2579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:16.492583 2579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.66
I20260808 11:38:16.502041 2579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:44797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.66.keytab
--principal=oryx/127.24.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:44823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.66
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:16.503146 2579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:16.504374 2579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:16.516739 2589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:16.516841 2590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:16.516707 2593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:16.517782 2579 server_base.cc:1034] running on GCE node
I20260808 11:38:16.518411 2579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:16.519269 2579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:16.520488 2579 hybrid_clock.cc:648] HybridClock initialized: now 1786189096520438 us; error 35 us; skew 500 ppm
I20260808 11:38:16.520776 2579 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:38:16 dist-test-slave-tc2s krb5kdc[2224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189096, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:16.523761 2579 init.cc:377] Logged in from keytab as oryx/127.24.76.66@KRBTEST.COM (short username oryx)
I20260808 11:38:16.525045 2579 webserver.cc:460] Webserver started at http://127.24.76.66:44073/ using document root <none> and password file <none>
I20260808 11:38:16.525658 2579 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:16.525775 2579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:16.526100 2579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:16.528869 2579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-1/data/instance:
uuid: "9b038a3b88794c9e8a63d670257a770d"
format_stamp: "Formatted at 2026-08-08 11:38:16 on dist-test-slave-tc2s"
server_key: "c1cd19eb98a894335be4e3501f92354b"
server_key_iv: "5768ad9b3da7578024ced0c04de1216c"
server_key_version: "encryptionkey@0"
I20260808 11:38:16.529624 2579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance:
uuid: "9b038a3b88794c9e8a63d670257a770d"
format_stamp: "Formatted at 2026-08-08 11:38:16 on dist-test-slave-tc2s"
server_key: "c1cd19eb98a894335be4e3501f92354b"
server_key_iv: "5768ad9b3da7578024ced0c04de1216c"
server_key_version: "encryptionkey@0"
I20260808 11:38:16.534578 2579 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260808 11:38:16.537813 2603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:16.539170 2579 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:38:16.539297 2579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-1/wal
uuid: "9b038a3b88794c9e8a63d670257a770d"
format_stamp: "Formatted at 2026-08-08 11:38:16 on dist-test-slave-tc2s"
server_key: "c1cd19eb98a894335be4e3501f92354b"
server_key_iv: "5768ad9b3da7578024ced0c04de1216c"
server_key_version: "encryptionkey@0"
I20260808 11:38:16.539455 2579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:16.559027 2579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:16.562726 2579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:16.562990 2579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:16.563863 2579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:38:16.565109 2579 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:38:16.565181 2579 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:16.565279 2579 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:38:16.565325 2579 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:16.592762 2579 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.66:44933
I20260808 11:38:16.592809 2772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.66:44933 every 8 connection(s)
I20260808 11:38:16.594027 2579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
I20260808 11:38:16.604353 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 2579
I20260808 11:38:16.604508 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance
I20260808 11:38:16.604929 24881 external_mini_cluster.cc:1428] Setting key ebe733c1b282be1971cec97a35b81f61
Aug 08 11:38:16 dist-test-slave-tc2s krb5kdc[2224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189096, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM
I20260808 11:38:16.612577 2575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:16.596378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:32849 (local address 127.24.76.126:44823)
0808 11:38:16.596600 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:16.596606 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:16.597709 (+ 1103us) server_negotiation.cc:407] Connection header received
0808 11:38:16.599647 (+ 1938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:16.599651 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:16.599719 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:16.599805 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:16.601358 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:16.602111 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:16.603536 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:16.603921 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:16.604008 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:16.607301 (+ 3293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:16.607306 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:16.607310 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:16.607340 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:16.609707 (+ 2367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:16.610409 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:16.610413 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:16.610415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:16.610546 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:16.610955 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:16.610959 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:16.610961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:16.611241 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:16.611377 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:16.612042 (+ 665us) server_negotiation.cc:299] Negotiation successful
0808 11:38:16.612270 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 11:38:16.613435 2775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:16.596869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.24.76.66:32849)
0808 11:38:16.597562 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:16.597599 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:16.599405 (+ 1806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:16.600012 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:16.600020 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:16.600659 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:16.601212 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:16.601231 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:16.602222 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:16.602226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:16.603425 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:16.603438 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:16.604107 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:16.604112 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:16.604301 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:16.604903 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:16.604929 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:16.607092 (+ 2163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:16.609838 (+ 2746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:16.609847 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:16.609861 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:16.610296 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:16.610652 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:16.610657 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:16.610661 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:16.610842 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:16.611367 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:16.611375 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:16.611680 (+ 305us) client_negotiation.cc:769] Sending connection context
0808 11:38:16.611902 (+ 222us) client_negotiation.cc:240] Negotiation successful
0808 11:38:16.612375 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":259,"threads_started":1}
I20260808 11:38:16.614773 2773 heartbeater.cc:347] Connected to a master server at 127.24.76.126:44823
I20260808 11:38:16.615037 2773 heartbeater.cc:464] Registering TS with master...
I20260808 11:38:16.615708 2773 heartbeater.cc:511] Master 127.24.76.126:44823 requested a full tablet report, sending...
I20260808 11:38:16.617048 2294 ts_manager.cc:194] Registered new tserver with Master: 9b038a3b88794c9e8a63d670257a770d (127.24.76.66:44933)
I20260808 11:38:16.617817 2294 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.76.66@KRBTEST.COM'} at 127.24.76.66:32849
WARNING: no policy specified for oryx/127.24.76.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.76.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.67.keytab.
Entry for principal oryx/127.24.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.67.keytab.
I20260808 11:38:16.654929 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.67:0
--local_ip_for_outbound_sockets=127.24.76.67
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:44823
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.67.keytab
--principal=oryx/127.24.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:44797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:38:16.970854 2781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:16.971124 2781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:16.971259 2781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:16.978276 2781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:16.978385 2781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:16.978543 2781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.67
I20260808 11:38:16.987900 2781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:44797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.67.keytab
--principal=oryx/127.24.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:44823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.67
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:16.989070 2781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:16.990372 2781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:17.002869 2789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:17.002975 2792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:17.003036 2788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:17.003906 2781 server_base.cc:1034] running on GCE node
I20260808 11:38:17.004477 2781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:17.005311 2781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:17.006505 2781 hybrid_clock.cc:648] HybridClock initialized: now 1786189097006457 us; error 38 us; skew 500 ppm
I20260808 11:38:17.006793 2781 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:38:17 dist-test-slave-tc2s krb5kdc[2224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189097, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:17.009693 2781 init.cc:377] Logged in from keytab as oryx/127.24.76.67@KRBTEST.COM (short username oryx)
I20260808 11:38:17.011073 2781 webserver.cc:460] Webserver started at http://127.24.76.67:37991/ using document root <none> and password file <none>
I20260808 11:38:17.011699 2781 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:17.011819 2781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:17.012125 2781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:17.014994 2781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-2/data/instance:
uuid: "c4d17c1c8b43495f81c16293bc8986f0"
format_stamp: "Formatted at 2026-08-08 11:38:17 on dist-test-slave-tc2s"
server_key: "b35414da7b76323cd4dc338f5bcc3da8"
server_key_iv: "01c071da219e3e9de6e4afcbbb59d13b"
server_key_version: "encryptionkey@0"
I20260808 11:38:17.015734 2781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance:
uuid: "c4d17c1c8b43495f81c16293bc8986f0"
format_stamp: "Formatted at 2026-08-08 11:38:17 on dist-test-slave-tc2s"
server_key: "b35414da7b76323cd4dc338f5bcc3da8"
server_key_iv: "01c071da219e3e9de6e4afcbbb59d13b"
server_key_version: "encryptionkey@0"
I20260808 11:38:17.020596 2781 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 11:38:17.023905 2801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:17.025091 2781 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 11:38:17.025231 2781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-2/wal
uuid: "c4d17c1c8b43495f81c16293bc8986f0"
format_stamp: "Formatted at 2026-08-08 11:38:17 on dist-test-slave-tc2s"
server_key: "b35414da7b76323cd4dc338f5bcc3da8"
server_key_iv: "01c071da219e3e9de6e4afcbbb59d13b"
server_key_version: "encryptionkey@0"
I20260808 11:38:17.025404 2781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:17.035974 2781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:17.039592 2781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:17.039856 2781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:17.040709 2781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:38:17.042299 2781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:38:17.042398 2781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:17.042569 2781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:38:17.042636 2781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260808 11:38:17.075099 2781 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.67:43707
I20260808 11:38:17.075136 2972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.67:43707 every 8 connection(s)
I20260808 11:38:17.076284 2781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
I20260808 11:38:17.083534 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 2781
I20260808 11:38:17.083700 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance
I20260808 11:38:17.084154 24881 external_mini_cluster.cc:1428] Setting key 997e3ef0515c1816fef619a571e61782
Aug 08 11:38:17 dist-test-slave-tc2s krb5kdc[2224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189097, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM
I20260808 11:38:17.094846 2575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:17.078660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:51961 (local address 127.24.76.126:44823)
0808 11:38:17.078840 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:17.078846 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:17.079960 (+ 1114us) server_negotiation.cc:407] Connection header received
0808 11:38:17.081701 (+ 1741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:17.081707 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:17.081783 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:17.081868 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:17.083323 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:17.084137 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:17.085973 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:17.086456 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:17.086558 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:17.089948 (+ 3390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:17.089955 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:17.089958 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:17.090001 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:17.092352 (+ 2351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:17.092970 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:17.092974 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:17.092976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:17.093106 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:17.093455 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:17.093458 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:17.093460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:17.093704 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:17.093817 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:17.094434 (+ 617us) server_negotiation.cc:299] Negotiation successful
0808 11:38:17.094601 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 11:38:17.095738 2979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:17.079142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.24.76.67:51961)
0808 11:38:17.079810 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:17.079847 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:17.081433 (+ 1586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:17.082123 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:17.082131 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:17.082617 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:17.083156 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:17.083173 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:17.084287 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:17.084291 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:17.085795 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:17.085818 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:17.086680 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:17.086688 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:17.086990 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:17.087626 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:17.087648 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:17.089719 (+ 2071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:17.092477 (+ 2758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:17.092483 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:17.092493 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:17.092885 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:17.093188 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:17.093192 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:17.093196 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:17.093377 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:17.093796 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:17.093802 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:17.094134 (+ 332us) client_negotiation.cc:769] Sending connection context
0808 11:38:17.094351 (+ 217us) client_negotiation.cc:240] Negotiation successful
0808 11:38:17.094714 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":260,"threads_started":1}
I20260808 11:38:17.096909 2974 heartbeater.cc:347] Connected to a master server at 127.24.76.126:44823
I20260808 11:38:17.097164 2974 heartbeater.cc:464] Registering TS with master...
I20260808 11:38:17.097940 2974 heartbeater.cc:511] Master 127.24.76.126:44823 requested a full tablet report, sending...
I20260808 11:38:17.099364 2294 ts_manager.cc:194] Registered new tserver with Master: c4d17c1c8b43495f81c16293bc8986f0 (127.24.76.67:43707)
I20260808 11:38:17.100137 2294 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.76.67@KRBTEST.COM'} at 127.24.76.67:51961
I20260808 11:38:17.111029 24881 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:38:17.133242 2572 heartbeater.cc:503] Master 127.24.76.126:44823 was elected leader, sending a full tablet report...
WARNING: no policy specified for oryx/127.24.76.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.76.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.76.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.125.keytab.
Entry for principal oryx/127.24.76.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.125.keytab.
I20260808 11:38:17.163337 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.125:42897
--webserver_interface=127.24.76.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.125.keytab
--principal=oryx/127.24.76.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:44797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:44823,127.24.76.125:42897
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:38:17.485636 2983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:17.485910 2983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:17.486044 2983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:17.493068 2983 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:38:17.493162 2983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:17.493239 2983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:17.493310 2983 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:38:17.493381 2983 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:38:17.505854 2983 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:44797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:44823,127.24.76.125:42897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.125.keytab
--principal=oryx/127.24.76.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.125:42897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.76.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
W20260808 11:38:17.506228 2983 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.24.76.126:44823,127.24.76.125:42897'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 11:38:17.507396 2983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:17.508898 2983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:17.519920 2991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:17.519961 2994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:17.519989 2990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:17.520934 2983 server_base.cc:1034] running on GCE node
I20260808 11:38:17.521656 2983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:17.522927 2983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:17.524128 2983 hybrid_clock.cc:648] HybridClock initialized: now 1786189097524079 us; error 27 us; skew 500 ppm
I20260808 11:38:17.524405 2983 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:38:17 dist-test-slave-tc2s krb5kdc[2224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189097, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:17.527457 2983 init.cc:377] Logged in from keytab as oryx/127.24.76.125@KRBTEST.COM (short username oryx)
I20260808 11:38:17.528937 2983 webserver.cc:460] Webserver started at http://127.24.76.125:44869/ using document root <none> and password file <none>
I20260808 11:38:17.529629 2983 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:17.529716 2983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:17.530022 2983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:17.532959 2983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-1/data/instance:
uuid: "79a1352c611a496ba0577480a04e026f"
format_stamp: "Formatted at 2026-08-08 11:38:17 on dist-test-slave-tc2s"
server_key: "a71cec9a5d1aa13bd71ef4ce3159e2e5"
server_key_iv: "0c30bf5f8855d92886278d9ee4e7aebc"
server_key_version: "encryptionkey@0"
I20260808 11:38:17.533730 2983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-1/wal/instance:
uuid: "79a1352c611a496ba0577480a04e026f"
format_stamp: "Formatted at 2026-08-08 11:38:17 on dist-test-slave-tc2s"
server_key: "a71cec9a5d1aa13bd71ef4ce3159e2e5"
server_key_iv: "0c30bf5f8855d92886278d9ee4e7aebc"
server_key_version: "encryptionkey@0"
I20260808 11:38:17.538754 2983 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260808 11:38:17.542049 3003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:17.543493 2983 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 11:38:17.543653 2983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-1/wal
uuid: "79a1352c611a496ba0577480a04e026f"
format_stamp: "Formatted at 2026-08-08 11:38:17 on dist-test-slave-tc2s"
server_key: "a71cec9a5d1aa13bd71ef4ce3159e2e5"
server_key_iv: "0c30bf5f8855d92886278d9ee4e7aebc"
server_key_version: "encryptionkey@0"
I20260808 11:38:17.543848 2983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:17.571470 2983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:17.578370 2983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:17.579943 2983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:17.597414 2983 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.125:42897
I20260808 11:38:17.597476 3068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.125:42897 every 8 connection(s)
I20260808 11:38:17.598841 2983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-1/data/info.pb
I20260808 11:38:17.602368 3069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:17.602993 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 2983
I20260808 11:38:17.611696 3069 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:38:17 dist-test-slave-tc2s krb5kdc[2224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.76.125@KRBTEST.COM
I20260808 11:38:17.620139 2773 heartbeater.cc:503] Master 127.24.76.126:44823 was elected leader, sending a full tablet report...
Aug 08 11:38:17 dist-test-slave-tc2s krb5kdc[2224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189097, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.125@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM
I20260808 11:38:17.627424 3073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:17.605409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56006 (local address 127.24.76.125:42897)
0808 11:38:17.606290 (+ 881us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:17.606305 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:17.606694 (+ 389us) server_negotiation.cc:407] Connection header received
0808 11:38:17.608582 (+ 1888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:17.608609 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:17.609073 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:17.609498 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:17.610992 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:17.612951 (+ 1959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:17.614258 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:17.615105 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:17.615277 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:17.619503 (+ 4226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:17.619511 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:17.619521 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:17.619554 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:17.623526 (+ 3972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:17.624246 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:17.624252 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:17.624257 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:17.624398 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:17.624801 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:17.624805 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:17.624807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:17.625270 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:17.625592 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:17.625975 (+ 383us) server_negotiation.cc:299] Negotiation successful
0808 11:38:17.626292 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":369,"threads_started":1}
I20260808 11:38:17.629606 3078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:17.614438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.0.0.1:36960)
0808 11:38:17.615554 (+ 1116us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:17.615593 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:17.616007 (+ 414us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:17.616384 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:17.616388 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:17.616938 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:17.617190 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:17.617199 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:17.618494 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:17.618500 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:17.619565 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:17.619578 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:17.620529 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:17.620535 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:17.620719 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:17.621312 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:17.621341 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:17.623767 (+ 2426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:17.626872 (+ 3105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:17.626882 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:17.626888 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:17.627441 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:17.627835 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:17.627840 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:17.627844 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:17.628002 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:17.628627 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:17.628634 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:17.628816 (+ 182us) client_negotiation.cc:769] Sending connection context
0808 11:38:17.629000 (+ 184us) client_negotiation.cc:240] Negotiation successful
0808 11:38:17.629252 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":873,"thread_start_us":725,"threads_started":1}
I20260808 11:38:17.630106 3077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:17.614415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36960 (local address 127.24.76.126:44823)
0808 11:38:17.614973 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:17.614979 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:17.615700 (+ 721us) server_negotiation.cc:407] Connection header received
0808 11:38:17.616139 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:17.616145 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:17.616230 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:17.616342 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:17.617342 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:17.618343 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:17.619709 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:17.620343 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:17.620467 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:17.623964 (+ 3497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:17.623970 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:17.623975 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:17.624013 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:17.626748 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:17.627553 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:17.627558 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:17.627563 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:17.627721 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:17.628106 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:17.628111 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:17.628115 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:17.628500 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:17.628631 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:17.629197 (+ 566us) server_negotiation.cc:299] Negotiation successful
0808 11:38:17.629416 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":250,"threads_started":1}
I20260808 11:38:17.631623 3069 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:38:17 dist-test-slave-tc2s krb5kdc[2224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189097, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.125@KRBTEST.COM for oryx/127.24.76.125@KRBTEST.COM
I20260808 11:38:17.642514 3078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:17.632191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:42897 (local address 127.0.0.1:56010)
0808 11:38:17.632375 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:17.632392 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:17.632465 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:17.632697 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:17.632701 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:17.632980 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:17.633102 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:17.633110 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:17.634004 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:17.634007 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:17.634897 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:17.634910 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:17.635527 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:17.635531 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:17.635643 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:17.636062 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:17.636076 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:17.637836 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:17.640367 (+ 2531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:17.640372 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:17.640376 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:17.640728 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:17.641032 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:17.641037 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:17.641041 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:17.641156 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:17.641624 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:17.641628 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:17.641760 (+ 132us) client_negotiation.cc:769] Sending connection context
0808 11:38:17.641868 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 11:38:17.642153 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 11:38:17.642647 3073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:17.632342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56010 (local address 127.24.76.125:42897)
0808 11:38:17.632473 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:17.632477 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:17.632490 (+ 13us) server_negotiation.cc:407] Connection header received
0808 11:38:17.632557 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:17.632560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:17.632608 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:17.632675 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:17.633209 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:17.633849 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:17.635011 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:17.635382 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:17.635471 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:17.638012 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:17.638016 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:17.638019 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:17.638044 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:17.640240 (+ 2196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:17.640828 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:17.640832 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:17.640834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:17.640931 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:17.641249 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:17.641253 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:17.641256 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:17.641519 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:17.641636 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:17.641990 (+ 354us) server_negotiation.cc:299] Negotiation successful
0808 11:38:17.642289 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":115,"server-negotiator.queue_time_us":43}
I20260808 11:38:17.646986 3069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f: Bootstrap starting.
I20260808 11:38:17.649827 3069 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:17.650869 3069 log.cc:824] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f: Log is configured to *not* fsync() on all Append() calls
I20260808 11:38:17.653322 3069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f: No bootstrap required, opened a new log
I20260808 11:38:17.660094 3069 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } }
I20260808 11:38:17.660406 3069 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:17.660496 3069 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79a1352c611a496ba0577480a04e026f, State: Initialized, Role: FOLLOWER
I20260808 11:38:17.660986 3069 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } }
I20260808 11:38:17.661857 3082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } } }
I20260808 11:38:17.662256 3082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:38:17.662760 3069 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [sys.catalog]: configured and running, proceeding with master startup.
W20260808 11:38:17.670790 3101 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 11:38:17.670894 3101 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 11:38:17.674319 2983 master_runner.cc:428] Detected that this master 79a1352c611a496ba0577480a04e026f is joining an existing cluster
I20260808 11:38:17.674399 2983 master_runner.cc:432] Initiating AddMaster RPC to add 127.24.76.125:42897
I20260808 11:38:17.693764 3073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:17.678419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56016 (local address 127.24.76.125:42897)
0808 11:38:17.678625 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:17.678630 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:17.679391 (+ 761us) server_negotiation.cc:407] Connection header received
0808 11:38:17.679538 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:17.679544 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:17.679622 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:17.679739 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:17.680879 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:17.681945 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:17.684060 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:17.684725 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:17.684800 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:17.687666 (+ 2866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:17.687673 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:17.687677 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:17.687713 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:17.691150 (+ 3437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:17.691969 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:17.691973 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:17.691977 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:17.692114 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:17.692409 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:17.692413 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:17.692416 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:17.692712 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:17.692825 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:17.693212 (+ 387us) server_negotiation.cc:299] Negotiation successful
0808 11:38:17.693418 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 11:38:17.694743 3112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:17.678103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:42897 (local address 127.0.0.1:56016)
0808 11:38:17.679295 (+ 1192us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:17.679311 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:17.679417 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:17.679773 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:17.679777 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:17.680426 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:17.680751 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:17.680763 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:17.682140 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:17.682145 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:17.683920 (+ 1775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:17.683934 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:17.685082 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:17.685091 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:17.685293 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:17.686140 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:17.686220 (+ 80us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:17.687491 (+ 1271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:17.691268 (+ 3777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:17.691273 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:17.691277 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:17.691870 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:17.692210 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:17.692214 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:17.692217 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:17.692336 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:17.692889 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:17.692892 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:17.693014 (+ 122us) client_negotiation.cc:769] Sending connection context
0808 11:38:17.694021 (+ 1007us) client_negotiation.cc:240] Negotiation successful
0808 11:38:17.694290 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1076,"mutex_wait_us":109,"spinlock_wait_cycles":23424,"thread_start_us":888,"threads_started":1}
I20260808 11:38:17.695742 3077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:17.677812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36970 (local address 127.24.76.126:44823)
0808 11:38:17.678038 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:17.678045 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:17.678726 (+ 681us) server_negotiation.cc:407] Connection header received
0808 11:38:17.679013 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:17.679020 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:17.679115 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:17.679187 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:17.680729 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:17.681831 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:17.684081 (+ 2250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:17.684701 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:17.684797 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:17.687682 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:17.687688 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:17.687692 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:17.687726 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:17.691120 (+ 3394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:17.693498 (+ 2378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:17.693504 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:17.693508 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:17.693658 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:17.694129 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:17.694134 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:17.694137 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:17.694500 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:17.694640 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:17.695194 (+ 554us) server_negotiation.cc:299] Negotiation successful
0808 11:38:17.695425 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 11:38:17.695904 3111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:17.677976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.0.0.1:36970)
0808 11:38:17.678627 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:17.678644 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:17.678836 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:17.679403 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:17.679408 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:17.680107 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:17.680542 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:17.680555 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:17.682032 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:17.682037 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:17.683915 (+ 1878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:17.683934 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:17.685113 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:17.685120 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:17.685307 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:17.686123 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:17.686145 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:17.687487 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:17.691268 (+ 3781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:17.691273 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:17.691277 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:17.693363 (+ 2086us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:17.693801 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:17.693805 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:17.693808 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:17.693999 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:17.694728 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:17.694736 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:17.694940 (+ 204us) client_negotiation.cc:769] Sending connection context
0808 11:38:17.695059 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 11:38:17.695358 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"mutex_wait_us":167,"spinlock_wait_cycles":29056,"thread_start_us":651,"threads_started":1}
Aug 08 11:38:17 dist-test-slave-tc2s krb5kdc[2224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189095, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.126@KRBTEST.COM for oryx/127.24.76.125@KRBTEST.COM
I20260808 11:38:17.714905 3073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:17.701568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56024 (local address 127.24.76.125:42897)
0808 11:38:17.701729 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:17.701735 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:17.702533 (+ 798us) server_negotiation.cc:407] Connection header received
0808 11:38:17.702844 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:17.702850 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:17.702930 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:17.703015 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:17.704054 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:17.704734 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:17.705982 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:17.706463 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:17.706574 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:17.709507 (+ 2933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:17.709511 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:17.709514 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:17.709542 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:17.712120 (+ 2578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:17.712762 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:17.712767 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:17.712769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:17.712891 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:17.713344 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:17.713349 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:17.713353 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:17.713644 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:17.713774 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:17.714387 (+ 613us) server_negotiation.cc:299] Negotiation successful
0808 11:38:17.714597 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
WARNING: no policy specified for oryx/127.24.76.124@KRBTEST.COM; defaulting to no policy
I20260808 11:38:17.715652 3114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:17.701562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:42897 (local address 127.0.0.1:56024)
0808 11:38:17.702415 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:17.702449 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:17.702657 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:17.703089 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:17.703096 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:17.703708 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:17.703906 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:17.703915 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:17.704857 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:17.704863 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:17.705832 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:17.705844 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:17.706618 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:17.706624 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:17.706809 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:17.707348 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:17.707364 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:17.709337 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:17.712244 (+ 2907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:17.712256 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:17.712264 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:17.712650 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:17.713055 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:17.713061 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:17.713064 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:17.713247 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:17.713775 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:17.713783 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:17.714021 (+ 238us) client_negotiation.cc:769] Sending connection context
0808 11:38:17.715088 (+ 1067us) client_negotiation.cc:240] Negotiation successful
0808 11:38:17.715349 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":488,"threads_started":1}
I20260808 11:38:17.717819 2294 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.24.76.125:42897: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "79a1352c611a496ba0577480a04e026f"
member_type: NON_VOTER
last_known_addr {
host: "127.24.76.125"
port: 42897
}
attrs {
promote: true
}
}
dest_uuid: "6ca9db23e8f945df88bffa6f2836ea9a"
cas_config_opid_index: -1
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.76.124@KRBTEST.COM" created.
I20260808 11:38:17.719249 2294 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: NON_VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: true } }
I20260808 11:38:17.730218 3116 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 79a1352c611a496ba0577480a04e026f (127.24.76.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: NON_VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: true } } }
I20260808 11:38:17.731200 3036 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:17.735143 3036 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [term 1 FOLLOWER]: Refusing update from remote peer 6ca9db23e8f945df88bffa6f2836ea9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 11:38:17.734853 3116 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.24.76.125:42897
I20260808 11:38:17.736897 2983 master.cc:537] Master@127.24.76.125:42897 shutting down...
I20260808 11:38:17.746994 3116 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: NON_VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:38:17.747381 3118 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: NON_VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: true } } }
I20260808 11:38:17.747802 3118 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [sys.catalog]: This master's current role is: LEADER
W20260808 11:38:17.750013 2265 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260808 11:38:17.753530 2983 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:38:17.753975 2983 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:38:17.754189 2983 tablet_replica.cc:330] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f: stopping tablet replica
W20260808 11:38:17.755528 2265 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a -> Peer 79a1352c611a496ba0577480a04e026f (127.24.76.125:42897): Couldn't send request to peer 79a1352c611a496ba0577480a04e026f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.76.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.124.keytab.
Entry for principal oryx/127.24.76.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.124.keytab.
I20260808 11:38:17.764117 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.124:45267
--webserver_interface=127.24.76.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.124.keytab
--principal=oryx/127.24.76.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:44797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:44823,127.24.76.125:42897,127.24.76.124:45267
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:38:17.771061 2983 master.cc:552] Master@127.24.76.125:42897 shutdown complete.
I20260808 11:38:17.771188 2983 master_runner.cc:305] Clearing existing system tablet
I20260808 11:38:17.771644 2983 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:17.771785 2983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:17.774821 2983 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 11:38:17.776476 3129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:17.776815 2983 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:17.777087 2983 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260808 11:38:17.777170 2983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-1/wal
uuid: "79a1352c611a496ba0577480a04e026f"
format_stamp: "Formatted at 2026-08-08 11:38:17 on dist-test-slave-tc2s"
server_key: "a71cec9a5d1aa13bd71ef4ce3159e2e5"
server_key_iv: "0c30bf5f8855d92886278d9ee4e7aebc"
server_key_version: "encryptionkey@0"
I20260808 11:38:17.782533 2983 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:38:17.789565 2983 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 11:38:17.789743 2983 log.cc:1196] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 11:38:17.790202 2983 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f: Deleting consensus metadata
I20260808 11:38:17.790524 2983 master_runner.cc:315] Copying system tablet from 127.24.76.126:44823
I20260808 11:38:17.793485 2983 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f: tablet copy: Beginning tablet copy session from remote peer at address 127.24.76.126:44823
I20260808 11:38:17.808830 3077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:17.794562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36976 (local address 127.24.76.126:44823)
0808 11:38:17.794763 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:17.794771 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:17.795217 (+ 446us) server_negotiation.cc:407] Connection header received
0808 11:38:17.795428 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:17.795434 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:17.795524 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:17.795628 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:17.796641 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:17.797712 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:17.799610 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:17.800264 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:17.800400 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:17.802644 (+ 2244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:17.802651 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:17.802655 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:17.802693 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:17.805966 (+ 3273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:17.806735 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:17.806741 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:17.806744 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:17.806908 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:17.807318 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:17.807322 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:17.807325 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:17.807671 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:17.807826 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:17.808239 (+ 413us) server_negotiation.cc:299] Negotiation successful
0808 11:38:17.808459 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 11:38:17.808924 3143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:17.794532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.0.0.1:36976)
0808 11:38:17.795095 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:17.795115 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:17.795268 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:17.795677 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:17.795682 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:17.796214 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:17.796489 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:17.796502 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:17.797882 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:17.797934 (+ 52us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:17.799453 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:17.799469 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:17.800437 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:17.800443 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:17.800624 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:17.801295 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:17.801332 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:17.802434 (+ 1102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:17.806101 (+ 3667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:17.806107 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:17.806111 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:17.806609 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:17.807028 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:17.807033 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:17.807037 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:17.807212 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:17.807800 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:17.807806 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:17.807984 (+ 178us) client_negotiation.cc:769] Sending connection context
0808 11:38:17.808136 (+ 152us) client_negotiation.cc:240] Negotiation successful
0808 11:38:17.808451 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":296,"threads_started":1}
I20260808 11:38:17.810199 2326 tablet_copy_service.cc:140] P 6ca9db23e8f945df88bffa6f2836ea9a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 79a1352c611a496ba0577480a04e026f ({username='oryx', principal='oryx/127.24.76.125@KRBTEST.COM'} at 127.0.0.1:36976)
I20260808 11:38:17.810676 2326 tablet_copy_service.cc:161] P 6ca9db23e8f945df88bffa6f2836ea9a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 79a1352c611a496ba0577480a04e026f at {username='oryx', principal='oryx/127.24.76.125@KRBTEST.COM'} at 127.0.0.1:36976: session id = 79a1352c611a496ba0577480a04e026f-00000000000000000000000000000000
I20260808 11:38:17.815562 2326 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 11:38:17.818377 2983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 11:38:17.825522 2983 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f: tablet copy: Starting download of 0 data blocks...
I20260808 11:38:17.825958 2983 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f: tablet copy: Starting download of 1 WAL segments...
I20260808 11:38:17.829909 2983 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 11:38:17.834542 2326 tablet_copy_service.cc:342] P 6ca9db23e8f945df88bffa6f2836ea9a: Request end of tablet copy session 79a1352c611a496ba0577480a04e026f-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.24.76.125@KRBTEST.COM'} at 127.0.0.1:36976
I20260808 11:38:17.834818 2326 tablet_copy_service.cc:433] P 6ca9db23e8f945df88bffa6f2836ea9a: ending tablet copy session 79a1352c611a496ba0577480a04e026f-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 79a1352c611a496ba0577480a04e026f
I20260808 11:38:17.845952 2983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:17.848423 3145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:17.858631 3147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:17.858697 2983 server_base.cc:1034] running on GCE node
W20260808 11:38:17.859010 3149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:17.859411 2983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:17.860029 2983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:17.861218 2983 hybrid_clock.cc:648] HybridClock initialized: now 1786189097861163 us; error 33 us; skew 500 ppm
I20260808 11:38:17.861416 2983 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:38:17 dist-test-slave-tc2s krb5kdc[2224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189097, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:17.863237 2983 init.cc:377] Logged in from keytab as oryx/127.24.76.125@KRBTEST.COM (short username oryx)
I20260808 11:38:17.864732 2983 webserver.cc:460] Webserver started at http://127.24.76.125:35617/ using document root <none> and password file <none>
I20260808 11:38:17.865049 2983 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:17.865119 2983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:17.867391 2983 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 11:38:17.868978 3160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:17.869336 2983 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 11:38:17.869470 2983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-1/wal
uuid: "79a1352c611a496ba0577480a04e026f"
format_stamp: "Formatted at 2026-08-08 11:38:17 on dist-test-slave-tc2s"
server_key: "a71cec9a5d1aa13bd71ef4ce3159e2e5"
server_key_iv: "0c30bf5f8855d92886278d9ee4e7aebc"
server_key_version: "encryptionkey@0"
I20260808 11:38:17.869617 2983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:17.895313 2983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:17.898463 2983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:17.920195 2983 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.125:42897
I20260808 11:38:17.920215 3229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.125:42897 every 8 connection(s)
I20260808 11:38:17.921082 2983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-1/data/info.pb
I20260808 11:38:17.926652 3230 sys_catalog.cc:263] Verifying existing consensus state
I20260808 11:38:17.929796 3230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f: Bootstrap starting.
I20260808 11:38:17.944065 3230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 11:38:17.944512 3230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f: Bootstrap complete.
I20260808 11:38:17.945331 3230 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: NON_VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: true } }
I20260808 11:38:17.945569 3230 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 79a1352c611a496ba0577480a04e026f, State: Initialized, Role: LEARNER
I20260808 11:38:17.945788 3230 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: NON_VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: true } }
I20260808 11:38:17.946521 3235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: NON_VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: true } } }
I20260808 11:38:17.946802 3230 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:38:17.946967 3235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [sys.catalog]: This master's current role is: LEARNER
I20260808 11:38:17.955706 3253 catalog_manager.cc:1259] Loaded cluster ID: 76425cb76ac444de9d8e156385ebe8f6
I20260808 11:38:17.955785 3253 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f: loading cluster ID for follower catalog manager: success
I20260808 11:38:17.958151 3253 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f: acquiring CA information for follower catalog manager: success
I20260808 11:38:17.960301 3253 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 11:38:18.102542 2974 heartbeater.cc:503] Master 127.24.76.126:44823 was elected leader, sending a full tablet report...
I20260808 11:38:18.175822 3114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:18.165664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:42897 (local address 127.0.0.1:56040)
0808 11:38:18.165942 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:18.165963 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:18.166084 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:18.167550 (+ 1466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:18.167556 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:18.167595 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:38:18.167994 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:18.168009 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.170208 (+ 2199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.170214 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:18.172842 (+ 2628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:18.172859 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.174827 (+ 1968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.174833 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:18.175186 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:18.175250 (+ 64us) client_negotiation.cc:769] Sending connection context
0808 11:38:18.175357 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 11:38:18.175510 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134}
I20260808 11:38:18.176079 3254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:18.166066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56040 (local address 127.24.76.125:42897)
0808 11:38:18.167008 (+ 942us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:18.167014 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:18.167038 (+ 24us) server_negotiation.cc:407] Connection header received
0808 11:38:18.167109 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:18.167114 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:18.167236 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:18.167482 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:38:18.168166 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.169937 (+ 1771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.173015 (+ 3078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.174627 (+ 1612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.174744 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:18.175014 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:18.175414 (+ 400us) server_negotiation.cc:299] Negotiation successful
0808 11:38:18.175613 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":801,"thread_start_us":532,"threads_started":1}
I20260808 11:38:18.177516 3194 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260808 11:38:18.178190 3235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ca9db23e8f945df88bffa6f2836ea9a. Latest consensus state: current_term: 1 leader_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: NON_VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: true } } }
I20260808 11:38:18.178520 3235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [sys.catalog]: This master's current role is: LEARNER
W20260808 11:38:18.236181 3127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:18.236478 3127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:18.236647 3127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:18.246285 3127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:38:18.246505 3127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:18.246626 3127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:18.246785 3127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:38:18.246872 3127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:38:18.258381 3127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:44797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:44823,127.24.76.125:42897,127.24.76.124:45267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.124.keytab
--principal=oryx/127.24.76.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.124:45267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.76.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:18.259792 3127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:18.261087 3127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:18.273451 3260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:18.273980 3261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:18.274178 3264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:18.275163 3127 server_base.cc:1034] running on GCE node
I20260808 11:38:18.275826 3127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:18.277266 3127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:18.278486 3127 hybrid_clock.cc:648] HybridClock initialized: now 1786189098278367 us; error 73 us; skew 500 ppm
I20260808 11:38:18.278884 3127 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:38:18 dist-test-slave-tc2s krb5kdc[2224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189098, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:18.282357 3127 init.cc:377] Logged in from keytab as oryx/127.24.76.124@KRBTEST.COM (short username oryx)
I20260808 11:38:18.284025 3127 webserver.cc:460] Webserver started at http://127.24.76.124:41109/ using document root <none> and password file <none>
I20260808 11:38:18.284780 3127 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:18.284952 3127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:18.285315 3127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:18.288676 3127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-2/data/instance:
uuid: "f7c33bdb0de24324977cff0e51bf1d47"
format_stamp: "Formatted at 2026-08-08 11:38:18 on dist-test-slave-tc2s"
server_key: "6f85efb8488cc570d3db1f667d342e7e"
server_key_iv: "ffe4d724b459df0f12617a77c1f0d8a1"
server_key_version: "encryptionkey@0"
I20260808 11:38:18.289466 3127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-2/wal/instance:
uuid: "f7c33bdb0de24324977cff0e51bf1d47"
format_stamp: "Formatted at 2026-08-08 11:38:18 on dist-test-slave-tc2s"
server_key: "6f85efb8488cc570d3db1f667d342e7e"
server_key_iv: "ffe4d724b459df0f12617a77c1f0d8a1"
server_key_version: "encryptionkey@0"
I20260808 11:38:18.295028 3127 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 11:38:18.298691 3273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:18.299921 3127 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260808 11:38:18.300077 3127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-2/wal
uuid: "f7c33bdb0de24324977cff0e51bf1d47"
format_stamp: "Formatted at 2026-08-08 11:38:18 on dist-test-slave-tc2s"
server_key: "6f85efb8488cc570d3db1f667d342e7e"
server_key_iv: "ffe4d724b459df0f12617a77c1f0d8a1"
server_key_version: "encryptionkey@0"
I20260808 11:38:18.300251 3127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:18.322322 3127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:18.327286 3127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:18.327649 3127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:18.359200 3127 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.124:45267
I20260808 11:38:18.359329 3353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.124:45267 every 8 connection(s)
I20260808 11:38:18.360755 3127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-2/data/info.pb
I20260808 11:38:18.365046 3357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:18.369957 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 3127
I20260808 11:38:18.374526 3357 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:38:18 dist-test-slave-tc2s krb5kdc[2224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.76.124@KRBTEST.COM
Aug 08 11:38:18 dist-test-slave-tc2s krb5kdc[2224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189098, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.124@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM
I20260808 11:38:18.397505 3365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:18.378878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36990 (local address 127.24.76.126:44823)
0808 11:38:18.380207 (+ 1329us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:18.380211 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:18.380226 (+ 15us) server_negotiation.cc:407] Connection header received
0808 11:38:18.380274 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:18.380278 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:18.380347 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:18.380556 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:18.382599 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.383568 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.385658 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.386197 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.386299 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:18.391270 (+ 4971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:18.391274 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:18.391278 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:18.391310 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:18.394268 (+ 2958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:18.395110 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:18.395115 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:18.395118 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:18.395276 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:18.395774 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:18.395777 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:18.395779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:18.396036 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:18.396142 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:18.396902 (+ 760us) server_negotiation.cc:299] Negotiation successful
0808 11:38:18.397110 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1229,"thread_start_us":1125,"threads_started":1}
I20260808 11:38:18.398761 3363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:18.378460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.0.0.1:36990)
0808 11:38:18.379594 (+ 1134us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:18.379636 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:18.379861 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:18.380666 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:18.380673 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:18.381806 (+ 1133us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:18.382412 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:18.382429 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.383744 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.383748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:18.385502 (+ 1754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:18.385517 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.386455 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.386464 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:18.386649 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:18.387308 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:18.387345 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:18.391058 (+ 3713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:18.394382 (+ 3324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:18.394391 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:18.394399 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:18.394995 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:18.395377 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:18.395381 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:18.395385 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:18.395672 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:18.396152 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:18.396160 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:18.396410 (+ 250us) client_negotiation.cc:769] Sending connection context
0808 11:38:18.396807 (+ 397us) client_negotiation.cc:240] Negotiation successful
0808 11:38:18.397239 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":780,"mutex_wait_us":117,"spinlock_wait_cycles":14720,"thread_start_us":531,"threads_started":1}
I20260808 11:38:18.398859 3362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:18.372342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39438 (local address 127.24.76.124:45267)
0808 11:38:18.373418 (+ 1076us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:18.373431 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:18.373487 (+ 56us) server_negotiation.cc:407] Connection header received
0808 11:38:18.375891 (+ 2404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:18.375914 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:18.376324 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:18.376845 (+ 521us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:18.379221 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.381391 (+ 2170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.383399 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.384437 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.384564 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:18.390509 (+ 5945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:18.390522 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:18.390536 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:18.390576 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:18.394005 (+ 3429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:18.394771 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:18.394780 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:18.394783 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:18.395036 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:18.395376 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:18.395381 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:18.395385 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:18.396090 (+ 705us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:18.396391 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:18.396689 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 11:38:18.397126 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":818,"spinlock_wait_cycles":19456,"thread_start_us":558,"threads_started":1}
I20260808 11:38:18.400905 3357 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:38:18 dist-test-slave-tc2s krb5kdc[2224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189098, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.124@KRBTEST.COM for oryx/127.24.76.125@KRBTEST.COM
I20260808 11:38:18.414734 3363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:18.401748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:42897 (local address 127.0.0.1:56052)
0808 11:38:18.401944 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:18.401963 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:18.402057 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:18.402365 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:18.402369 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:18.402691 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:18.402816 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:18.402824 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.403929 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.403933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:18.405010 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:18.405022 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.405716 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.405722 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:18.405947 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:18.406545 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:18.406568 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:18.408677 (+ 2109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:18.412126 (+ 3449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:18.412132 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:18.412136 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:18.412629 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:18.413048 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:18.413053 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:18.413057 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:18.413183 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:18.413689 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:18.413694 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:18.413822 (+ 128us) client_negotiation.cc:769] Sending connection context
0808 11:38:18.413981 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 11:38:18.414258 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 11:38:18.414841 3254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:18.401838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56052 (local address 127.24.76.125:42897)
0808 11:38:18.402040 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:18.402045 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:18.402093 (+ 48us) server_negotiation.cc:407] Connection header received
0808 11:38:18.402155 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:18.402160 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:18.402221 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:18.402311 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:18.402993 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.403800 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.405149 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.405583 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.405693 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:18.408871 (+ 3178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:18.408876 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:18.408879 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:18.408909 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:18.412009 (+ 3100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:18.412737 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:18.412741 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:18.412743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:18.412940 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:18.413282 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:18.413286 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:18.413288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:18.413578 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:18.413707 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:18.414084 (+ 377us) server_negotiation.cc:299] Negotiation successful
0808 11:38:18.414332 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260808 11:38:18.416124 3357 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.76.124" port: 45267 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:38:18 dist-test-slave-tc2s krb5kdc[2224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189098, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.124@KRBTEST.COM for oryx/127.24.76.124@KRBTEST.COM
I20260808 11:38:18.429379 3363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:18.416996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:45267 (local address 127.0.0.1:39444)
0808 11:38:18.417152 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:18.417169 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:18.417246 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:18.417484 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:18.417487 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:18.417870 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:18.418077 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:18.418087 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.419035 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.419039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:18.419990 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:18.420002 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.420796 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.420800 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:18.421067 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:18.421522 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:18.421539 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:18.423559 (+ 2020us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:18.426942 (+ 3383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:18.426946 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:18.426949 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:18.427370 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:18.427758 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:18.427761 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:18.427764 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:18.427954 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:18.428464 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:18.428468 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:18.428618 (+ 150us) client_negotiation.cc:769] Sending connection context
0808 11:38:18.428819 (+ 201us) client_negotiation.cc:240] Negotiation successful
0808 11:38:18.428993 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 11:38:18.429560 3362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:18.417057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39444 (local address 127.24.76.124:45267)
0808 11:38:18.417206 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:18.417210 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:18.417225 (+ 15us) server_negotiation.cc:407] Connection header received
0808 11:38:18.417321 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:18.417324 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:18.417374 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:18.417441 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:18.418189 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.418915 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.420098 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.420662 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.420752 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:18.423743 (+ 2991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:18.423748 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:18.423751 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:18.423778 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:18.426833 (+ 3055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:18.427471 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:18.427474 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:18.427476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:18.427636 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:18.428065 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:18.428070 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:18.428073 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:18.428367 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:18.428482 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:18.428927 (+ 445us) server_negotiation.cc:299] Negotiation successful
0808 11:38:18.429116 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":53}
I20260808 11:38:18.435211 3357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47: Bootstrap starting.
I20260808 11:38:18.439450 3357 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:18.441047 3357 log.cc:824] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47: Log is configured to *not* fsync() on all Append() calls
I20260808 11:38:18.444589 3357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47: No bootstrap required, opened a new log
I20260808 11:38:18.453341 3357 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } } peers { permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 45267 } }
I20260808 11:38:18.453691 3357 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:18.453856 3357 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7c33bdb0de24324977cff0e51bf1d47, State: Initialized, Role: FOLLOWER
I20260808 11:38:18.454483 3357 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } } peers { permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 45267 } }
I20260808 11:38:18.455382 3367 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } } peers { permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 45267 } } }
I20260808 11:38:18.455724 3367 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:38:18.456525 3357 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 11:38:18.467943 3381 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 11:38:18.468096 3381 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 11:38:18.471678 3127 master_runner.cc:428] Detected that this master f7c33bdb0de24324977cff0e51bf1d47 is joining an existing cluster
I20260808 11:38:18.471771 3127 master_runner.cc:432] Initiating AddMaster RPC to add 127.24.76.124:45267
I20260808 11:38:18.505139 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 2390
I20260808 11:38:18.513343 3254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:18.476119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56058 (local address 127.24.76.125:42897)
0808 11:38:18.476302 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:18.476308 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:18.495481 (+ 19173us) server_negotiation.cc:407] Connection header received
0808 11:38:18.495695 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:18.495699 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:18.495774 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:18.495836 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:18.498041 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.499165 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.501638 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.502392 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.502465 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:18.504812 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:18.504818 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:18.504822 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:18.504860 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:18.508709 (+ 3849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:18.510663 (+ 1954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:18.510669 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:18.510672 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:18.510836 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:18.511374 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:18.511379 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:18.511382 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:18.511780 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:18.511918 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:18.512676 (+ 758us) server_negotiation.cc:299] Negotiation successful
0808 11:38:18.512910 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 11:38:18.514633 3395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:18.477305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:42897 (local address 127.0.0.1:56058)
0808 11:38:18.495274 (+ 17969us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:18.495290 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:18.495562 (+ 272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:18.496032 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:18.496036 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:18.497204 (+ 1168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:18.497820 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:18.497833 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.499322 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.499327 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:18.501470 (+ 2143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:18.501485 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.502700 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.502704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:18.502864 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:18.503561 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:18.503585 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:18.504574 (+ 989us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:18.508827 (+ 4253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:18.508832 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:18.508837 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:18.510511 (+ 1674us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:18.511001 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:18.511006 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:18.511009 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:18.511231 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:18.512004 (+ 773us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:18.512013 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:18.512327 (+ 314us) client_negotiation.cc:769] Sending connection context
0808 11:38:18.513599 (+ 1272us) client_negotiation.cc:240] Negotiation successful
0808 11:38:18.513939 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17858,"mutex_wait_us":93,"thread_start_us":16619,"threads_started":1}
I20260808 11:38:18.525172 3398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:18.476888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.0.0.1:36998)
0808 11:38:18.494648 (+ 17760us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:18.494667 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:18.494936 (+ 269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:18.495450 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:18.495462 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:18.496389 (+ 927us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:18.497127 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:18.497142 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.501994 (+ 4852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.501999 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:18.506678 (+ 4679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:18.506693 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.510044 (+ 3351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.510054 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:18.510276 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:18.516573 (+ 6297us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:18.516594 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:18.517399 (+ 805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:18.521860 (+ 4461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:18.521865 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:18.521868 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:18.522451 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:18.522948 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:18.522953 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:18.522957 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:18.523147 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:18.523906 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:18.523915 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:18.524218 (+ 303us) client_negotiation.cc:769] Sending connection context
0808 11:38:18.524365 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 11:38:18.524669 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17397,"thread_start_us":9375,"threads_started":1}
I20260808 11:38:18.526089 3365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:18.476206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36998 (local address 127.24.76.126:44823)
0808 11:38:18.476386 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:18.476391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:18.494869 (+ 18478us) server_negotiation.cc:407] Connection header received
0808 11:38:18.495063 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:18.495069 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:18.495157 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:18.495260 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:18.497329 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.498671 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.506842 (+ 8171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.507826 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.507938 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:18.517589 (+ 9651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:18.517595 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:18.517600 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:18.517639 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:18.521713 (+ 4074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:18.522602 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:18.522607 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:18.522611 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:18.522806 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:18.523282 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:18.523287 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:18.523290 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:18.523673 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:18.523833 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:18.525462 (+ 1629us) server_negotiation.cc:299] Negotiation successful
0808 11:38:18.525672 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Aug 08 11:38:18 dist-test-slave-tc2s krb5kdc[2224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.76.65@KRBTEST.COM
I20260808 11:38:18.538511 3399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:18.506389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34012 (local address 127.24.76.65:33613)
0808 11:38:18.509557 (+ 3168us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:18.509564 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:18.509591 (+ 27us) server_negotiation.cc:407] Connection header received
0808 11:38:18.509654 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:18.509660 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:18.509849 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:18.519508 (+ 9659us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:18.519925 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.521637 (+ 1712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.523945 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.525007 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.525141 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:18.530701 (+ 5560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:18.530716 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:18.530735 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:18.530781 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:18.535075 (+ 4294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:18.535929 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:18.535935 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:18.535937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:18.536074 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:18.536608 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:18.536614 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:18.536617 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:18.537073 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:18.537394 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:18.537784 (+ 390us) server_negotiation.cc:299] Negotiation successful
0808 11:38:18.538104 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2887,"thread_start_us":2723,"threads_started":1}
I20260808 11:38:18.540041 2565 generic_service.cc:196] Checking for leaks (request via RPC)
Aug 08 11:38:18 dist-test-slave-tc2s krb5kdc[2224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189095, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.126@KRBTEST.COM for oryx/127.24.76.124@KRBTEST.COM
I20260808 11:38:18.551664 3394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:18.476949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:45267 (local address 127.0.0.1:39448)
0808 11:38:18.495186 (+ 18237us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:18.495204 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:18.495504 (+ 300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:18.495946 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:18.495951 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:18.497163 (+ 1212us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:18.497678 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:18.497693 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.499322 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.499327 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:18.501470 (+ 2143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:18.501502 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.510179 (+ 8677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.510191 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:18.510371 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:18.511411 (+ 1040us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:18.511432 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:18.512821 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:18.548050 (+ 35229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:18.548055 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:18.548059 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:18.548655 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:18.549208 (+ 553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:18.549212 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:18.549217 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:18.549410 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:18.550277 (+ 867us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:18.550282 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:18.550479 (+ 197us) client_negotiation.cc:769] Sending connection context
0808 11:38:18.550661 (+ 182us) client_negotiation.cc:240] Negotiation successful
0808 11:38:18.551068 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18100,"mutex_wait_us":96,"thread_start_us":16968,"threads_started":1}
I20260808 11:38:18.553207 3362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:18.476146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39448 (local address 127.24.76.124:45267)
0808 11:38:18.476317 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:18.476323 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:18.495402 (+ 19079us) server_negotiation.cc:407] Connection header received
0808 11:38:18.495626 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:18.495631 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:18.495687 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:18.495755 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:18.497845 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.499108 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.501611 (+ 2503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.505432 (+ 3821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.505554 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:18.512988 (+ 7434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:18.512995 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:18.512999 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:18.513031 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:18.547923 (+ 34892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:18.548803 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:18.548809 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:18.548812 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:18.549072 (+ 260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:18.549552 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:18.549557 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:18.549560 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:18.550012 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:18.550157 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:18.552599 (+ 2442us) server_negotiation.cc:299] Negotiation successful
0808 11:38:18.552818 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 11:38:18.554193 3114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:18.534404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:45267 (local address 127.0.0.1:39450)
0808 11:38:18.534731 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:18.534751 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:18.534863 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:18.536269 (+ 1406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:18.536274 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:18.536707 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:18.536917 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:18.536927 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.538576 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.538581 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:18.539754 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:18.539768 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.540756 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.540763 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:18.540907 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:18.541533 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:18.541551 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:18.545940 (+ 4389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:18.550752 (+ 4812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:18.550757 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:18.550761 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:18.551372 (+ 611us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:18.551936 (+ 564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:18.551941 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:18.551945 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:18.552123 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:18.552782 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:18.552788 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:18.553038 (+ 250us) client_negotiation.cc:769] Sending connection context
0808 11:38:18.553213 (+ 175us) client_negotiation.cc:240] Negotiation successful
0808 11:38:18.553790 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177}
I20260808 11:38:18.555429 3400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:18.534467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39450 (local address 127.24.76.124:45267)
0808 11:38:18.535679 (+ 1212us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:18.535684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:18.535724 (+ 40us) server_negotiation.cc:407] Connection header received
0808 11:38:18.535819 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:18.535824 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:18.535932 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:18.536246 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:18.537091 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.538450 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.539893 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.540540 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.541209 (+ 669us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:18.546191 (+ 4982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:18.546199 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:18.546204 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:18.546244 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:18.550608 (+ 4364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:18.551536 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:18.551541 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:18.551545 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:18.551701 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:18.552270 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:18.552275 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:18.552279 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:18.552646 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:18.554558 (+ 1912us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:18.554689 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 11:38:18.554878 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1045,"thread_start_us":812,"threads_started":1}
I20260808 11:38:18.557006 2294 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.24.76.124:45267: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47"
member_type: NON_VOTER
last_known_addr {
host: "127.24.76.124"
port: 45267
}
attrs {
promote: true
}
}
dest_uuid: "6ca9db23e8f945df88bffa6f2836ea9a"
cas_config_opid_index: 5
I20260808 11:38:18.558161 2294 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: NON_VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: true } } peers { permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: NON_VOTER last_known_addr { host: "127.24.76.124" port: 45267 } attrs { promote: true } }
I20260808 11:38:18.560163 3118 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f7c33bdb0de24324977cff0e51bf1d47 (127.24.76.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: NON_VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: true } } peers { permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: NON_VOTER last_known_addr { host: "127.24.76.124" port: 45267 } attrs { promote: true } } }
I20260808 11:38:18.563987 3118 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.24.76.124:45267
I20260808 11:38:18.564733 3127 master.cc:537] Master@127.24.76.124:45267 shutting down...
I20260808 11:38:18.564975 3312 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:18.566128 3194 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [term 1 LEARNER]: Refusing update from remote peer 6ca9db23e8f945df88bffa6f2836ea9a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 11:38:18.567526 3404 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: NON_VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 11:38:18.567801 3403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: NON_VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: true } } peers { permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: NON_VOTER last_known_addr { host: "127.24.76.124" port: 45267 } attrs { promote: true } } }
I20260808 11:38:18.568207 3403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [sys.catalog]: This master's current role is: LEADER
I20260808 11:38:18.571943 3312 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [term 1 FOLLOWER]: Refusing update from remote peer 6ca9db23e8f945df88bffa6f2836ea9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 11:38:18.573555 3403 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: NON_VOTER last_known_addr { host: "127.24.76.124" port: 45267 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 11:38:18.580215 3286 tablet.cc:2369] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:38:18.582224 3127 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:38:18.582602 3127 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:38:18.582794 3127 tablet_replica.cc:330] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47: stopping tablet replica
I20260808 11:38:18.586217 3194 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f7c33bdb0de24324977cff0e51bf1d47 (127.24.76.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: NON_VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: true } } peers { permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: NON_VOTER last_known_addr { host: "127.24.76.124" port: 45267 } attrs { promote: true } } }
I20260808 11:38:18.590641 3235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: NON_VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: true } } peers { permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: NON_VOTER last_known_addr { host: "127.24.76.124" port: 45267 } attrs { promote: true } } }
I20260808 11:38:18.590952 3235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [sys.catalog]: This master's current role is: LEARNER
I20260808 11:38:18.591403 3403 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a: attempting to promote NON_VOTER 79a1352c611a496ba0577480a04e026f to VOTER
I20260808 11:38:18.592262 3403 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: false } } peers { permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: NON_VOTER last_known_addr { host: "127.24.76.124" port: 45267 } attrs { promote: true } }
I20260808 11:38:18.593588 3127 master.cc:552] Master@127.24.76.124:45267 shutdown complete.
I20260808 11:38:18.593668 3127 master_runner.cc:305] Clearing existing system tablet
I20260808 11:38:18.594077 3127 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:18.594170 3127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:18.594883 3194 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [term 1 LEARNER]: Refusing update from remote peer 6ca9db23e8f945df88bffa6f2836ea9a: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 11:38:18.595428 3118 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260808 11:38:18.596935 2267 connection.cc:698] client connection to 127.24.76.124:45267 send error: Network error: sendmsg error: Broken pipe (error 32)
I20260808 11:38:18.597836 3404 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 79a1352c611a496ba0577480a04e026f (127.24.76.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: false } } peers { permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: NON_VOTER last_known_addr { host: "127.24.76.124" port: 45267 } attrs { promote: true } } }
I20260808 11:38:18.598739 3114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:18.598138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:45267 (local address 127.0.0.1:39458)
0808 11:38:18.598285 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:18.598395 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 11:38:18.599084 2267 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a -> Peer f7c33bdb0de24324977cff0e51bf1d47 (127.24.76.124:45267): Couldn't send request to peer f7c33bdb0de24324977cff0e51bf1d47. Status: Network error: Client connection negotiation failed: client connection to 127.24.76.124:45267: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 11:38:18.603334 3193 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 79a1352c611a496ba0577480a04e026f (127.24.76.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: false } } peers { permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: NON_VOTER last_known_addr { host: "127.24.76.124" port: 45267 } attrs { promote: true } } }
I20260808 11:38:18.607473 3235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: false } } peers { permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: NON_VOTER last_known_addr { host: "127.24.76.124" port: 45267 } attrs { promote: true } } }
I20260808 11:38:18.607717 3235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:38:18.614126 3403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: false } } peers { permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: NON_VOTER last_known_addr { host: "127.24.76.124" port: 45267 } attrs { promote: true } } }
I20260808 11:38:18.614409 3403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [sys.catalog]: This master's current role is: LEADER
I20260808 11:38:18.614861 3127 fs_manager.cc:709] Time spent opening directory manager: real 0.020s user 0.002s sys 0.015s
I20260808 11:38:18.614773 3404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: false } } peers { permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: NON_VOTER last_known_addr { host: "127.24.76.124" port: 45267 } attrs { promote: true } } }
I20260808 11:38:18.615049 3404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [sys.catalog]: This master's current role is: LEADER
I20260808 11:38:18.616809 3413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:18.617385 3127 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:18.617677 3127 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260808 11:38:18.617801 3127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-2/wal
uuid: "f7c33bdb0de24324977cff0e51bf1d47"
format_stamp: "Formatted at 2026-08-08 11:38:18 on dist-test-slave-tc2s"
server_key: "6f85efb8488cc570d3db1f667d342e7e"
server_key_iv: "ffe4d724b459df0f12617a77c1f0d8a1"
server_key_version: "encryptionkey@0"
I20260808 11:38:18.624105 3127 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:38:18.631675 3127 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 11:38:18.631819 3127 log.cc:1196] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 11:38:18.632298 3127 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47: Deleting consensus metadata
I20260808 11:38:18.632575 3127 master_runner.cc:315] Copying system tablet from 127.24.76.126:44823
I20260808 11:38:18.635787 3127 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47: tablet copy: Beginning tablet copy session from remote peer at address 127.24.76.126:44823
I20260808 11:38:18.652937 3365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:18.636617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37008 (local address 127.24.76.126:44823)
0808 11:38:18.636819 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:18.636825 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:18.637383 (+ 558us) server_negotiation.cc:407] Connection header received
0808 11:38:18.637577 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:18.637583 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:18.637667 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:18.637775 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:18.638836 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.640105 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.642287 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.643254 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.643417 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:18.645993 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:18.646000 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:18.646005 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:18.646045 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:18.649736 (+ 3691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:18.650630 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:18.650636 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:18.650640 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:18.650839 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:18.651315 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:18.651320 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:18.651323 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:18.651761 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:18.651910 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:18.652383 (+ 473us) server_negotiation.cc:299] Negotiation successful
0808 11:38:18.652602 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260808 11:38:18.653383 3423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:18.636567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.0.0.1:37008)
0808 11:38:18.637255 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:18.637279 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:18.637422 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:18.637816 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:18.637822 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:18.638398 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:18.638666 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:18.638680 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.640270 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.640279 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:18.642122 (+ 1843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:18.642138 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:18.643416 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:18.643426 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:18.643607 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:18.644441 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:18.644463 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:18.645760 (+ 1297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:18.649921 (+ 4161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:18.649926 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:18.649930 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:18.650465 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:18.650995 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:18.650999 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:18.651003 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:18.651199 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:18.651884 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:18.651889 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:18.652081 (+ 192us) client_negotiation.cc:769] Sending connection context
0808 11:38:18.652241 (+ 160us) client_negotiation.cc:240] Negotiation successful
0808 11:38:18.652568 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":440,"threads_started":1}
I20260808 11:38:18.654049 2326 tablet_copy_service.cc:140] P 6ca9db23e8f945df88bffa6f2836ea9a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f7c33bdb0de24324977cff0e51bf1d47 ({username='oryx', principal='oryx/127.24.76.124@KRBTEST.COM'} at 127.0.0.1:37008)
I20260808 11:38:18.654217 2326 tablet_copy_service.cc:161] P 6ca9db23e8f945df88bffa6f2836ea9a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f7c33bdb0de24324977cff0e51bf1d47 at {username='oryx', principal='oryx/127.24.76.124@KRBTEST.COM'} at 127.0.0.1:37008: session id = f7c33bdb0de24324977cff0e51bf1d47-00000000000000000000000000000000
I20260808 11:38:18.658116 2326 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 11:38:18.660074 3127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 11:38:18.668210 3127 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47: tablet copy: Starting download of 0 data blocks...
I20260808 11:38:18.668632 3127 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47: tablet copy: Starting download of 1 WAL segments...
I20260808 11:38:18.671461 3127 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 11:38:18.676174 2326 tablet_copy_service.cc:342] P 6ca9db23e8f945df88bffa6f2836ea9a: Request end of tablet copy session f7c33bdb0de24324977cff0e51bf1d47-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.24.76.124@KRBTEST.COM'} at 127.0.0.1:37008
I20260808 11:38:18.676344 2326 tablet_copy_service.cc:433] P 6ca9db23e8f945df88bffa6f2836ea9a: ending tablet copy session f7c33bdb0de24324977cff0e51bf1d47-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f7c33bdb0de24324977cff0e51bf1d47
I20260808 11:38:18.683784 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 2390
I20260808 11:38:18.692844 3127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:18.695261 3424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:18.701083 3425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:18.711580 3428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:18.712007 3127 server_base.cc:1034] running on GCE node
I20260808 11:38:18.712292 3127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:18.712833 3127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:18.714066 3127 hybrid_clock.cc:648] HybridClock initialized: now 1786189098714027 us; error 30 us; skew 500 ppm
I20260808 11:38:18.714274 3127 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:38:18.715699 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.65:33613
--local_ip_for_outbound_sockets=127.24.76.65
--tserver_master_addrs=127.24.76.126:44823
--webserver_port=46385
--webserver_interface=127.24.76.65
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.65.keytab
--principal=oryx/127.24.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:44797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.76.126:44823,127.24.76.125:42897
--tserver_master_addrs=127.24.76.126:44823,127.24.76.125:42897,127.24.76.124:45267 with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 11:38:18 dist-test-slave-tc2s krb5kdc[2224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189098, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:18.717235 3127 init.cc:377] Logged in from keytab as oryx/127.24.76.124@KRBTEST.COM (short username oryx)
I20260808 11:38:18.718537 3127 webserver.cc:460] Webserver started at http://127.24.76.124:35855/ using document root <none> and password file <none>
I20260808 11:38:18.718850 3127 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:18.718953 3127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:18.721861 3127 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260808 11:38:18.723733 3438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:18.724128 3127 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260808 11:38:18.724242 3127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-2/wal
uuid: "f7c33bdb0de24324977cff0e51bf1d47"
format_stamp: "Formatted at 2026-08-08 11:38:18 on dist-test-slave-tc2s"
server_key: "6f85efb8488cc570d3db1f667d342e7e"
server_key_iv: "ffe4d724b459df0f12617a77c1f0d8a1"
server_key_version: "encryptionkey@0"
I20260808 11:38:18.724409 3127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:18.737841 3127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:18.742084 3127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:18.775575 3127 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.124:45267
I20260808 11:38:18.775638 3530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.124:45267 every 8 connection(s)
I20260808 11:38:18.776647 3127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/master-2/data/info.pb
I20260808 11:38:18.781872 3531 sys_catalog.cc:263] Verifying existing consensus state
I20260808 11:38:18.785769 3531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47: Bootstrap starting.
I20260808 11:38:18.803996 3531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 11:38:18.804508 3531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47: Bootstrap complete.
I20260808 11:38:18.805634 3531 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: false } } peers { permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: NON_VOTER last_known_addr { host: "127.24.76.124" port: 45267 } attrs { promote: true } }
I20260808 11:38:18.805953 3531 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f7c33bdb0de24324977cff0e51bf1d47, State: Initialized, Role: LEARNER
I20260808 11:38:18.806228 3531 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: false } } peers { permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: NON_VOTER last_known_addr { host: "127.24.76.124" port: 45267 } attrs { promote: true } }
I20260808 11:38:18.807339 3531 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:38:18.807097 3534 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: false } } peers { permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: NON_VOTER last_known_addr { host: "127.24.76.124" port: 45267 } attrs { promote: true } } }
I20260808 11:38:18.807618 3534 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [sys.catalog]: This master's current role is: LEARNER
I20260808 11:38:18.816200 3552 catalog_manager.cc:1259] Loaded cluster ID: 76425cb76ac444de9d8e156385ebe8f6
I20260808 11:38:18.816298 3552 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47: loading cluster ID for follower catalog manager: success
I20260808 11:38:18.819679 3552 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47: acquiring CA information for follower catalog manager: success
I20260808 11:38:18.822301 3552 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 11:38:19.119668 3445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:19.119941 3445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:19.120157 3445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 11:38:19.121934 3563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:19.114791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39466 (local address 127.24.76.124:45267)
0808 11:38:19.115500 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:19.115507 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:19.115684 (+ 177us) server_negotiation.cc:407] Connection header received
0808 11:38:19.115906 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:19.115912 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:19.116018 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:19.116286 (+ 268us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:38:19.116620 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:19.118250 (+ 1630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:19.120069 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:19.120889 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:19.121014 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:19.121232 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:19.121470 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 11:38:19.121602 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":444,"threads_started":1}
I20260808 11:38:19.122030 3562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:19.114649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:45267 (local address 127.0.0.1:39466)
0808 11:38:19.115459 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:19.115484 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:19.115746 (+ 262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:19.116293 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:19.116297 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:19.116320 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:38:19.116495 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:19.116503 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:19.118382 (+ 1879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:19.118386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:19.119910 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:19.119928 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:19.121095 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:19.121102 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:19.121312 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:19.121338 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 11:38:19.121419 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 11:38:19.121553 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":348,"threads_started":1}
I20260808 11:38:19.123099 3486 raft_consensus.cc:1203] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 11:38:19.123750 3534 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ca9db23e8f945df88bffa6f2836ea9a. Latest consensus state: current_term: 1 leader_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: false } } peers { permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: NON_VOTER last_known_addr { host: "127.24.76.124" port: 45267 } attrs { promote: true } } }
I20260808 11:38:19.124058 3534 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [sys.catalog]: This master's current role is: LEARNER
W20260808 11:38:19.128194 3445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:19.128320 3445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:19.128567 3445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.65
I20260808 11:38:19.139052 3445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:44797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.65.keytab
--principal=oryx/127.24.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.65:33613
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.76.65
--webserver_port=46385
--tserver_master_addrs=127.24.76.126:44823,127.24.76.125:42897,127.24.76.124:45267
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.65
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:19.140478 3445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:19.142175 3445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:19.156177 3568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:19.156242 3569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:19.156209 3571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:19.159209 3445 server_base.cc:1034] running on GCE node
I20260808 11:38:19.159901 3445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:19.160822 3445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:19.162003 3445 hybrid_clock.cc:648] HybridClock initialized: now 1786189099161900 us; error 98 us; skew 500 ppm
I20260808 11:38:19.162304 3445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:38:19 dist-test-slave-tc2s krb5kdc[2224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189099, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:19.165802 3445 init.cc:377] Logged in from keytab as oryx/127.24.76.65@KRBTEST.COM (short username oryx)
I20260808 11:38:19.167266 3445 webserver.cc:460] Webserver started at http://127.24.76.65:46385/ using document root <none> and password file <none>
I20260808 11:38:19.167977 3445 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:19.168109 3445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:19.173292 3445 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 11:38:19.175905 3579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:19.177211 3445 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 11:38:19.177366 3445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-0/wal
uuid: "900623cd4f4a40c99098e6dfbe9f684f"
format_stamp: "Formatted at 2026-08-08 11:38:16 on dist-test-slave-tc2s"
server_key: "12eea97d396a69a65e3afdc857947489"
server_key_iv: "f6790a7f5f73f0a5d0a3e2160f0e541f"
server_key_version: "encryptionkey@0"
I20260808 11:38:19.178457 3445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:19.216048 3445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:19.220534 3445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:19.220789 3445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:19.221812 3445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:38:19.223361 3445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:38:19.223456 3445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:19.223565 3445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:38:19.223636 3445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:19.271291 3445 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.65:33613
I20260808 11:38:19.271382 3748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.65:33613 every 8 connection(s)
I20260808 11:38:19.272766 3445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
I20260808 11:38:19.278658 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 3445
I20260808 11:38:19.279086 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 2579
Aug 08 11:38:19 dist-test-slave-tc2s krb5kdc[2224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.76.66@KRBTEST.COM
Aug 08 11:38:19 dist-test-slave-tc2s krb5kdc[2224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189099, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.125@KRBTEST.COM
Aug 08 11:38:19 dist-test-slave-tc2s krb5kdc[2224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189099, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM
Aug 08 11:38:19 dist-test-slave-tc2s krb5kdc[2224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189099, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.124@KRBTEST.COM
I20260808 11:38:19.322500 3756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:19.276221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:43673 (local address 127.24.76.126:44823)
0808 11:38:19.291583 (+ 15362us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:19.291589 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:19.300340 (+ 8751us) server_negotiation.cc:407] Connection header received
0808 11:38:19.300415 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:19.300420 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:19.300509 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:19.300665 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:19.303367 (+ 2702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:19.304543 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:19.307670 (+ 3127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:19.308358 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:19.308477 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:19.314087 (+ 5610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:19.314093 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:19.314097 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:19.314141 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:19.317933 (+ 3792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:19.319192 (+ 1259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:19.319197 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:19.319200 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:19.319360 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:19.319926 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:19.319931 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:19.319934 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:19.320323 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:19.320497 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:19.321755 (+ 1258us) server_negotiation.cc:299] Negotiation successful
0808 11:38:19.322003 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15226,"thread_start_us":15364,"threads_started":1}
I20260808 11:38:19.322747 3760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:19.280490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39178 (local address 127.24.76.66:44933)
0808 11:38:19.294517 (+ 14027us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:19.294525 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:19.294682 (+ 157us) server_negotiation.cc:407] Connection header received
0808 11:38:19.294883 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:19.294893 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:19.295204 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:19.295377 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:19.298003 (+ 2626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:19.300190 (+ 2187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:19.302346 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:19.303341 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:19.303454 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:19.311453 (+ 7999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:19.311470 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:19.311487 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:19.311532 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:19.318163 (+ 6631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:19.319208 (+ 1045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:19.319216 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:19.319219 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:19.319465 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:19.319957 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:19.319961 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:19.319964 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:19.320436 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:19.320760 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:19.322054 (+ 1294us) server_negotiation.cc:299] Negotiation successful
0808 11:38:19.322304 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13814,"thread_start_us":13660,"threads_started":1}
I20260808 11:38:19.322803 3755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:19.275688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:57669 (local address 127.24.76.125:42897)
0808 11:38:19.291065 (+ 15377us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:19.291073 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:19.300671 (+ 9598us) server_negotiation.cc:407] Connection header received
0808 11:38:19.300805 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:19.300811 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:19.300910 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:19.301153 (+ 243us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:19.303349 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:19.304525 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:19.307661 (+ 3136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:19.308369 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:19.308479 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:19.313557 (+ 5078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:19.313562 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:19.313565 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:19.313600 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:19.318170 (+ 4570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:19.319227 (+ 1057us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:19.319232 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:19.319236 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:19.319411 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:19.319965 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:19.319971 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:19.319974 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:19.320398 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:19.320543 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:19.322095 (+ 1552us) server_negotiation.cc:299] Negotiation successful
0808 11:38:19.322313 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15226,"thread_start_us":15384,"threads_started":1}
I20260808 11:38:19.323171 3563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:19.275758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:51177 (local address 127.24.76.124:45267)
0808 11:38:19.275940 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:19.275946 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:19.295846 (+ 19900us) server_negotiation.cc:407] Connection header received
0808 11:38:19.299194 (+ 3348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:19.299202 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:19.299319 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:19.299436 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:19.303351 (+ 3915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:19.304573 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:19.307662 (+ 3089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:19.308369 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:19.308479 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:19.315719 (+ 7240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:19.315726 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:19.315730 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:19.315771 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:19.320119 (+ 4348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:19.320896 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:19.320901 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:19.320905 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:19.321059 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:19.321545 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:19.321550 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:19.321553 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:19.321961 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:19.322099 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:19.322605 (+ 506us) server_negotiation.cc:299] Negotiation successful
0808 11:38:19.322834 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 11:38:19.324333 2767 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:19.325701 3758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:19.277952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:42897 (local address 127.24.76.65:57669)
0808 11:38:19.300502 (+ 22550us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:19.300518 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:19.300693 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:19.301341 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:19.301345 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:19.302245 (+ 900us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:19.303189 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:19.303199 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:19.304741 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:19.304746 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:19.307493 (+ 2747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:19.307508 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:19.309590 (+ 2082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:19.309600 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:19.309796 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:19.310395 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:19.310417 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:19.313297 (+ 2880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:19.318340 (+ 5043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:19.318345 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:19.318348 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:19.319067 (+ 719us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:19.319581 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:19.319586 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:19.319589 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:19.319825 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:19.320605 (+ 780us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:19.320609 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:19.321028 (+ 419us) client_negotiation.cc:769] Sending connection context
0808 11:38:19.321377 (+ 349us) client_negotiation.cc:240] Negotiation successful
0808 11:38:19.324222 (+ 2845us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22444,"mutex_wait_us":319,"spinlock_wait_cycles":10496,"thread_start_us":22204,"threads_started":1}
I20260808 11:38:19.325953 3757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:19.277549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.24.76.65:43673)
0808 11:38:19.299719 (+ 22170us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:19.299738 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:19.299863 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:19.301077 (+ 1214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:19.301086 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:19.302180 (+ 1094us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:19.303149 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:19.303170 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:19.304742 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:19.304747 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:19.307490 (+ 2743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:19.307507 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:19.308858 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:19.308865 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:19.309169 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:19.310164 (+ 995us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:19.310188 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:19.313857 (+ 3669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:19.318110 (+ 4253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:19.318119 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:19.318135 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:19.319040 (+ 905us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:19.319512 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:19.319516 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:19.319520 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:19.319788 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:19.320573 (+ 785us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:19.320580 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:19.321029 (+ 449us) client_negotiation.cc:769] Sending connection context
0808 11:38:19.321387 (+ 358us) client_negotiation.cc:240] Negotiation successful
0808 11:38:19.324577 (+ 3190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22016,"mutex_wait_us":2953,"spinlock_wait_cycles":8192,"thread_start_us":17528,"threads_started":1}
I20260808 11:38:19.325742 3763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:19.277504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:45267 (local address 127.24.76.65:51177)
0808 11:38:19.295665 (+ 18161us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:19.295729 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:19.298799 (+ 3070us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:19.301075 (+ 2276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:19.301086 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:19.302054 (+ 968us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:19.303135 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:19.303170 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:19.304746 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:19.304750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:19.307502 (+ 2752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:19.307511 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:19.308879 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:19.308888 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:19.309193 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:19.310110 (+ 917us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:19.310138 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:19.315491 (+ 5353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:19.320281 (+ 4790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:19.320299 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:19.320302 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:19.320765 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:19.321215 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:19.321219 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:19.321223 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:19.321424 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:19.322178 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:19.322182 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:19.322317 (+ 135us) client_negotiation.cc:769] Sending connection context
0808 11:38:19.323463 (+ 1146us) client_negotiation.cc:240] Negotiation successful
0808 11:38:19.324404 (+ 941us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17805,"mutex_wait_us":921,"spinlock_wait_cycles":44160,"thread_start_us":21804,"threads_started":1}
I20260808 11:38:19.327476 3751 heartbeater.cc:347] Connected to a master server at 127.24.76.126:44823
I20260808 11:38:19.327536 3752 heartbeater.cc:347] Connected to a master server at 127.24.76.125:42897
I20260808 11:38:19.327754 3750 heartbeater.cc:347] Connected to a master server at 127.24.76.124:45267
I20260808 11:38:19.327836 3751 heartbeater.cc:464] Registering TS with master...
I20260808 11:38:19.327869 3750 heartbeater.cc:464] Registering TS with master...
I20260808 11:38:19.327840 3752 heartbeater.cc:464] Registering TS with master...
I20260808 11:38:19.329099 3752 heartbeater.cc:511] Master 127.24.76.125:42897 requested a full tablet report, sending...
I20260808 11:38:19.329136 3750 heartbeater.cc:511] Master 127.24.76.124:45267 requested a full tablet report, sending...
I20260808 11:38:19.329190 3751 heartbeater.cc:511] Master 127.24.76.126:44823 requested a full tablet report, sending...
I20260808 11:38:19.331125 2294 ts_manager.cc:194] Re-registered known tserver with Master: 900623cd4f4a40c99098e6dfbe9f684f (127.24.76.65:33613)
I20260808 11:38:19.331424 3474 ts_manager.cc:194] Registered new tserver with Master: 900623cd4f4a40c99098e6dfbe9f684f (127.24.76.65:33613)
I20260808 11:38:19.331655 3181 ts_manager.cc:194] Registered new tserver with Master: 900623cd4f4a40c99098e6dfbe9f684f (127.24.76.65:33613)
I20260808 11:38:19.332306 2294 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.76.65@KRBTEST.COM'} at 127.24.76.65:43673
I20260808 11:38:19.416071 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 2579
I20260808 11:38:19.438277 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.66:44933
--local_ip_for_outbound_sockets=127.24.76.66
--tserver_master_addrs=127.24.76.126:44823
--webserver_port=44073
--webserver_interface=127.24.76.66
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.66.keytab
--principal=oryx/127.24.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:44797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.76.126:44823,127.24.76.125:42897
--tserver_master_addrs=127.24.76.126:44823,127.24.76.125:42897,127.24.76.124:45267 with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:38:19.711712 3404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: false } } peers { permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: NON_VOTER last_known_addr { host: "127.24.76.124" port: 45267 } attrs { promote: true } } }
I20260808 11:38:19.712070 3404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [sys.catalog]: This master's current role is: LEADER
I20260808 11:38:19.713851 3771 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a: attempting to promote NON_VOTER f7c33bdb0de24324977cff0e51bf1d47 to VOTER
I20260808 11:38:19.714526 3771 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: false } } peers { permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 45267 } attrs { promote: false } }
I20260808 11:38:19.716596 3486 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [term 1 LEARNER]: Refusing update from remote peer 6ca9db23e8f945df88bffa6f2836ea9a: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260808 11:38:19.716632 3193 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [term 1 FOLLOWER]: Refusing update from remote peer 6ca9db23e8f945df88bffa6f2836ea9a: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260808 11:38:19.717182 3770 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260808 11:38:19.717614 3771 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 45267 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260808 11:38:19.720386 3770 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, f7c33bdb0de24324977cff0e51bf1d47 (127.24.76.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: false } } peers { permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 45267 } attrs { promote: false } } }
I20260808 11:38:19.720997 3193 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, f7c33bdb0de24324977cff0e51bf1d47 (127.24.76.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: false } } peers { permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 45267 } attrs { promote: false } } }
I20260808 11:38:19.722255 3486 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, f7c33bdb0de24324977cff0e51bf1d47 (127.24.76.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: false } } peers { permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 45267 } attrs { promote: false } } }
I20260808 11:38:19.728788 3771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: false } } peers { permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 45267 } attrs { promote: false } } }
I20260808 11:38:19.729061 3771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [sys.catalog]: This master's current role is: LEADER
I20260808 11:38:19.729342 3771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: false } } peers { permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 45267 } attrs { promote: false } } }
I20260808 11:38:19.729632 3771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [sys.catalog]: This master's current role is: LEADER
I20260808 11:38:19.729961 3771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: false } } peers { permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 45267 } attrs { promote: false } } }
I20260808 11:38:19.730209 3771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a [sys.catalog]: This master's current role is: LEADER
I20260808 11:38:19.733970 3778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: false } } peers { permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 45267 } attrs { promote: false } } }
I20260808 11:38:19.734385 3778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:38:19.734683 3779 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: false } } peers { permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 45267 } attrs { promote: false } } }
I20260808 11:38:19.734948 3779 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 11:38:19.776769 3768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:19.777009 3768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:19.777127 3768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:19.784313 3768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:19.784430 3768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:19.784583 3768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.66
I20260808 11:38:19.793932 3768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:44797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.66.keytab
--principal=oryx/127.24.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.66:44933
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.76.66
--webserver_port=44073
--tserver_master_addrs=127.24.76.126:44823,127.24.76.125:42897,127.24.76.124:45267
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.66
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:19.795145 3768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:19.796386 3768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:19.809118 3787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:19.809195 3784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:19.809190 3790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:19.810271 3768 server_base.cc:1034] running on GCE node
I20260808 11:38:19.810894 3768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:19.811784 3768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:19.812976 3768 hybrid_clock.cc:648] HybridClock initialized: now 1786189099812924 us; error 47 us; skew 500 ppm
I20260808 11:38:19.813277 3768 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:38:19 dist-test-slave-tc2s krb5kdc[2224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189099, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:19.816262 3768 init.cc:377] Logged in from keytab as oryx/127.24.76.66@KRBTEST.COM (short username oryx)
I20260808 11:38:19.817564 3768 webserver.cc:460] Webserver started at http://127.24.76.66:44073/ using document root <none> and password file <none>
I20260808 11:38:19.818251 3768 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:19.818382 3768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:19.823594 3768 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 11:38:19.826439 3803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:19.827709 3768 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:38:19.827853 3768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-1/wal
uuid: "9b038a3b88794c9e8a63d670257a770d"
format_stamp: "Formatted at 2026-08-08 11:38:16 on dist-test-slave-tc2s"
server_key: "c1cd19eb98a894335be4e3501f92354b"
server_key_iv: "5768ad9b3da7578024ced0c04de1216c"
server_key_version: "encryptionkey@0"
I20260808 11:38:19.828846 3768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:19.850430 3768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:19.854467 3768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:19.854738 3768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:19.855661 3768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:38:19.856963 3768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:38:19.857057 3768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:19.857179 3768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:38:19.857255 3768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:19.886310 3768 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.66:44933
I20260808 11:38:19.886376 3975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.66:44933 every 8 connection(s)
I20260808 11:38:19.887657 3768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
I20260808 11:38:19.891410 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 3768
I20260808 11:38:19.891712 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 2781
Aug 08 11:38:19 dist-test-slave-tc2s krb5kdc[2224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189099, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.125@KRBTEST.COM
Aug 08 11:38:19 dist-test-slave-tc2s krb5kdc[2224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189099, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM
Aug 08 11:38:19 dist-test-slave-tc2s krb5kdc[2224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189099, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.124@KRBTEST.COM
Aug 08 11:38:19 dist-test-slave-tc2s krb5kdc[2224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.76.67@KRBTEST.COM
I20260808 11:38:19.959568 3982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:19.890381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:53985 (local address 127.24.76.124:45267)
0808 11:38:19.936672 (+ 46291us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:19.936678 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:19.936699 (+ 21us) server_negotiation.cc:407] Connection header received
0808 11:38:19.936762 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:19.936767 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:19.936856 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:19.937168 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:19.942317 (+ 5149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:19.943785 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:19.945959 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:19.946619 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:19.946750 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:19.952157 (+ 5407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:19.952165 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:19.952169 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:19.952211 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:19.955593 (+ 3382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:19.956701 (+ 1108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:19.956707 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:19.956710 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:19.956853 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:19.957416 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:19.957422 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:19.957425 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:19.957753 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:19.957880 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:19.958953 (+ 1073us) server_negotiation.cc:299] Negotiation successful
0808 11:38:19.959188 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46129,"thread_start_us":45994,"threads_started":1}
I20260808 11:38:19.959882 3983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:19.890495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:45581 (local address 127.24.76.125:42897)
0808 11:38:19.936895 (+ 46400us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:19.936901 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:19.936924 (+ 23us) server_negotiation.cc:407] Connection header received
0808 11:38:19.936999 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:19.937005 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:19.937095 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:19.937385 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:19.940792 (+ 3407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:19.942181 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:19.944651 (+ 2470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:19.945438 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:19.945566 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:19.952711 (+ 7145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:19.952718 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:19.952723 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:19.952762 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:19.956600 (+ 3838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:19.957489 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:19.957494 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:19.957497 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:19.957698 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:19.958233 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:19.958238 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:19.958241 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:19.958578 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:19.958729 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:19.959352 (+ 623us) server_negotiation.cc:299] Negotiation successful
0808 11:38:19.959555 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46267,"thread_start_us":46107,"threads_started":1}
I20260808 11:38:19.960788 3984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:19.890968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.24.76.66:52265)
0808 11:38:19.913066 (+ 22098us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:19.913123 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:19.915940 (+ 2817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:19.939188 (+ 23248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:19.939192 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:19.939647 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:19.940390 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:19.940416 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:19.941686 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:19.941692 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:19.944389 (+ 2697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:19.944408 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:19.946139 (+ 1731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:19.946145 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:19.946438 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:19.947245 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:19.947292 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:19.951554 (+ 4262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:19.956042 (+ 4488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:19.956047 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:19.956051 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:19.956600 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:19.957039 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:19.957044 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:19.957047 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:19.957365 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:19.958109 (+ 744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:19.958114 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:19.958471 (+ 357us) client_negotiation.cc:769] Sending connection context
0808 11:38:19.958815 (+ 344us) client_negotiation.cc:240] Negotiation successful
0808 11:38:19.959390 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21751,"mutex_wait_us":67,"spinlock_wait_cycles":28160,"thread_start_us":28126,"threads_started":1}
I20260808 11:38:19.961059 3985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:19.891043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:45267 (local address 127.24.76.66:53985)
0808 11:38:19.918733 (+ 27690us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:19.918753 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:19.918867 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:19.937747 (+ 18880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:19.937761 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:19.939830 (+ 2069us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:19.942121 (+ 2291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:19.942140 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:19.944000 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:19.944005 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:19.945758 (+ 1753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:19.945775 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:19.947025 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:19.947035 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:19.947297 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:19.948008 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:19.948031 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:19.951881 (+ 3850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:19.955756 (+ 3875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:19.955765 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:19.955782 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:19.956557 (+ 775us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:19.957005 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:19.957011 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:19.957014 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:19.957268 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:19.958000 (+ 732us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:19.958008 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:19.958471 (+ 463us) client_negotiation.cc:769] Sending connection context
0808 11:38:19.960120 (+ 1649us) client_negotiation.cc:240] Negotiation successful
0808 11:38:19.960418 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27562,"mutex_wait_us":3,"spinlock_wait_cycles":4608,"thread_start_us":43704,"threads_started":1}
I20260808 11:38:19.961684 3991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:19.892740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58666 (local address 127.24.76.67:43707)
0808 11:38:19.912231 (+ 19491us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:19.912241 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:19.917254 (+ 5013us) server_negotiation.cc:407] Connection header received
0808 11:38:19.917346 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:19.917355 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:19.917636 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:19.917787 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:19.938748 (+ 20961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:19.940517 (+ 1769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:19.942770 (+ 2253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:19.943666 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:19.943771 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:19.953245 (+ 9474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:19.953267 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:19.953286 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:19.953361 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:19.957563 (+ 4202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:19.958403 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:19.958411 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:19.958415 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:19.958595 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:19.959031 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:19.959035 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:19.959038 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:19.959429 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:19.959747 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:19.960839 (+ 1092us) server_negotiation.cc:299] Negotiation successful
0808 11:38:19.961098 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19284,"thread_start_us":19508,"threads_started":1}
I20260808 11:38:19.962024 3987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:19.890968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:42897 (local address 127.24.76.66:45581)
0808 11:38:19.934558 (+ 43590us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:19.934579 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:19.934724 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:19.938587 (+ 3863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:19.938591 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:19.939259 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:19.940618 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:19.940634 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:19.942472 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:19.942478 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:19.944463 (+ 1985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:19.944480 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:19.945961 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:19.945970 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:19.946264 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:19.947245 (+ 981us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:19.947282 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:19.952451 (+ 5169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:19.956735 (+ 4284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:19.956739 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:19.956742 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:19.957348 (+ 606us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:19.957849 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:19.957854 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:19.957857 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:19.958107 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:19.958791 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:19.958795 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:19.958944 (+ 149us) client_negotiation.cc:769] Sending connection context
0808 11:38:19.959089 (+ 145us) client_negotiation.cc:240] Negotiation successful
0808 11:38:19.960570 (+ 1481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43416,"mutex_wait_us":1539,"spinlock_wait_cycles":23040,"thread_start_us":27272,"threads_started":1}
I20260808 11:38:19.962095 3980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:19.890336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:52265 (local address 127.24.76.126:44823)
0808 11:38:19.937925 (+ 47589us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:19.937931 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:19.937951 (+ 20us) server_negotiation.cc:407] Connection header received
0808 11:38:19.938021 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:19.938027 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:19.938109 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:19.938333 (+ 224us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:19.940602 (+ 2269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:19.941512 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:19.944877 (+ 3365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:19.945728 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:19.945863 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:19.952198 (+ 6335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:19.952206 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:19.952210 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:19.952263 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:19.955884 (+ 3621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:19.956742 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:19.956747 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:19.956750 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:19.956906 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:19.957488 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:19.957493 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:19.957496 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:19.957878 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:19.958050 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:19.961326 (+ 3276us) server_negotiation.cc:299] Negotiation successful
0808 11:38:19.961585 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47434,"thread_start_us":47334,"threads_started":1}
I20260808 11:38:19.962716 3976 heartbeater.cc:347] Connected to a master server at 127.24.76.124:45267
I20260808 11:38:19.963061 3976 heartbeater.cc:464] Registering TS with master...
I20260808 11:38:19.963356 2968 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:19.964061 3978 heartbeater.cc:347] Connected to a master server at 127.24.76.125:42897
I20260808 11:38:19.964182 3977 heartbeater.cc:347] Connected to a master server at 127.24.76.126:44823
I20260808 11:38:19.964197 3976 heartbeater.cc:511] Master 127.24.76.124:45267 requested a full tablet report, sending...
I20260808 11:38:19.964334 3977 heartbeater.cc:464] Registering TS with master...
I20260808 11:38:19.964999 3977 heartbeater.cc:511] Master 127.24.76.126:44823 requested a full tablet report, sending...
I20260808 11:38:19.964222 3978 heartbeater.cc:464] Registering TS with master...
I20260808 11:38:19.965741 3978 heartbeater.cc:511] Master 127.24.76.125:42897 requested a full tablet report, sending...
I20260808 11:38:19.966326 3474 ts_manager.cc:194] Registered new tserver with Master: 9b038a3b88794c9e8a63d670257a770d (127.24.76.66:44933)
I20260808 11:38:19.966526 2294 ts_manager.cc:194] Re-registered known tserver with Master: 9b038a3b88794c9e8a63d670257a770d (127.24.76.66:44933)
I20260808 11:38:19.967509 2294 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.76.66@KRBTEST.COM'} at 127.24.76.66:52265
I20260808 11:38:19.968761 3181 ts_manager.cc:194] Registered new tserver with Master: 9b038a3b88794c9e8a63d670257a770d (127.24.76.66:44933)
I20260808 11:38:20.060798 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 2781
I20260808 11:38:20.083863 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.67:43707
--local_ip_for_outbound_sockets=127.24.76.67
--tserver_master_addrs=127.24.76.126:44823
--webserver_port=37991
--webserver_interface=127.24.76.67
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.67.keytab
--principal=oryx/127.24.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:44797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.76.126:44823,127.24.76.125:42897
--tserver_master_addrs=127.24.76.126:44823,127.24.76.125:42897,127.24.76.124:45267 with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:38:20.335078 3751 heartbeater.cc:503] Master 127.24.76.126:44823 was elected leader, sending a full tablet report...
W20260808 11:38:20.411741 3994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:20.412012 3994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:20.412143 3994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:20.419113 3994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:20.419222 3994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:20.419371 3994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.67
I20260808 11:38:20.428771 3994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:44797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.67.keytab
--principal=oryx/127.24.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.67:43707
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.76.67
--webserver_port=37991
--tserver_master_addrs=127.24.76.126:44823,127.24.76.125:42897,127.24.76.124:45267
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.67
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:20.430048 3994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:20.431298 3994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:20.443895 4007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:20.443939 4010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:20.443972 4005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:20.444796 3994 server_base.cc:1034] running on GCE node
I20260808 11:38:20.445398 3994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:20.446363 3994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:20.447573 3994 hybrid_clock.cc:648] HybridClock initialized: now 1786189100447524 us; error 36 us; skew 500 ppm
I20260808 11:38:20.447901 3994 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:38:20 dist-test-slave-tc2s krb5kdc[2224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189100, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:20.454594 3994 init.cc:377] Logged in from keytab as oryx/127.24.76.67@KRBTEST.COM (short username oryx)
I20260808 11:38:20.455973 3994 webserver.cc:460] Webserver started at http://127.24.76.67:37991/ using document root <none> and password file <none>
I20260808 11:38:20.456629 3994 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:20.456759 3994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:20.462018 3994 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:38:20.464901 4019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:20.466259 3994 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 11:38:20.466413 3994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-2/wal
uuid: "c4d17c1c8b43495f81c16293bc8986f0"
format_stamp: "Formatted at 2026-08-08 11:38:17 on dist-test-slave-tc2s"
server_key: "b35414da7b76323cd4dc338f5bcc3da8"
server_key_iv: "01c071da219e3e9de6e4afcbbb59d13b"
server_key_version: "encryptionkey@0"
I20260808 11:38:20.467350 3994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:20.483104 3994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:20.486750 3994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:20.487012 3994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:20.487921 3994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:38:20.489169 3994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:38:20.489260 3994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:20.489372 3994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:38:20.489447 3994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:20.518464 3994 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.67:43707
I20260808 11:38:20.518529 4188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.67:43707 every 8 connection(s)
I20260808 11:38:20.519858 3994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
I20260808 11:38:20.527599 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 3994
Aug 08 11:38:20 dist-test-slave-tc2s krb5kdc[2224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189100, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.125@KRBTEST.COM
Aug 08 11:38:20 dist-test-slave-tc2s krb5kdc[2224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189100, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.124@KRBTEST.COM
Aug 08 11:38:20 dist-test-slave-tc2s krb5kdc[2224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189100, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM
I20260808 11:38:20.579350 4205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:20.523526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:45267 (local address 127.24.76.67:52885)
0808 11:38:20.539646 (+ 16120us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:20.539666 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:20.539809 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:20.540393 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:20.540401 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:20.541216 (+ 815us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:20.541538 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:20.541548 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.543101 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.543105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:20.544665 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:20.544679 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.547491 (+ 2812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.547498 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:20.547733 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:20.549065 (+ 1332us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:20.549087 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:20.558711 (+ 9624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:20.568119 (+ 9408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:20.568129 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:20.568150 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:20.569344 (+ 1194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:20.571480 (+ 2136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:20.571484 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:20.571488 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:20.573310 (+ 1822us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:20.576815 (+ 3505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:20.576820 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:20.576979 (+ 159us) client_negotiation.cc:769] Sending connection context
0808 11:38:20.577144 (+ 165us) client_negotiation.cc:240] Negotiation successful
0808 11:38:20.577713 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15980,"mutex_wait_us":60,"thread_start_us":15426,"threads_started":1}
I20260808 11:38:20.580647 4202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:20.523000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:52885 (local address 127.24.76.124:45267)
0808 11:38:20.533585 (+ 10585us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:20.533592 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:20.539916 (+ 6324us) server_negotiation.cc:407] Connection header received
0808 11:38:20.539988 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:20.539993 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:20.540086 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:20.540183 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:20.541697 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.542949 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.544835 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.545607 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.545724 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:20.559925 (+ 14201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:20.559931 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:20.559936 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:20.559973 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:20.567911 (+ 7938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:20.569502 (+ 1591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:20.569509 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:20.569512 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:20.569687 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:20.575266 (+ 5579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:20.575272 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:20.575276 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:20.575675 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:20.575844 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:20.579898 (+ 4054us) server_negotiation.cc:299] Negotiation successful
0808 11:38:20.580140 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10445,"thread_start_us":10336,"threads_started":1}
I20260808 11:38:20.582067 4189 heartbeater.cc:347] Connected to a master server at 127.24.76.124:45267
I20260808 11:38:20.582476 4189 heartbeater.cc:464] Registering TS with master...
I20260808 11:38:20.583053 4199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:20.522686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:60349 (local address 127.24.76.126:44823)
0808 11:38:20.534806 (+ 12120us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:20.534812 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:20.539824 (+ 5012us) server_negotiation.cc:407] Connection header received
0808 11:38:20.540040 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:20.540044 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:20.540118 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:20.540189 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:20.541701 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.542908 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.547329 (+ 4421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.548016 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.548140 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:20.559905 (+ 11765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:20.559911 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:20.559915 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:20.559953 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:20.564286 (+ 4333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:20.570592 (+ 6306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:20.570598 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:20.570601 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:20.570921 (+ 320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:20.573567 (+ 2646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:20.573572 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:20.573575 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:20.573995 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:20.574144 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:20.582400 (+ 8256us) server_negotiation.cc:299] Negotiation successful
0808 11:38:20.582655 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11996,"thread_start_us":11863,"threads_started":1}
I20260808 11:38:20.583983 4201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:20.522964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:55379 (local address 127.24.76.125:42897)
0808 11:38:20.533867 (+ 10903us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:20.533874 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:20.533945 (+ 71us) server_negotiation.cc:407] Connection header received
0808 11:38:20.534023 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:20.534028 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:20.534117 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:20.534226 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:20.537246 (+ 3020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.538451 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.544322 (+ 5871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.544957 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.545054 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:20.552480 (+ 7426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:20.552487 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:20.552492 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:20.552563 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:20.574758 (+ 22195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:20.577746 (+ 2988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:20.577751 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:20.577754 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:20.578008 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:20.578462 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:20.578467 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:20.578470 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:20.578861 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:20.579009 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:20.580266 (+ 1257us) server_negotiation.cc:299] Negotiation successful
0808 11:38:20.580571 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10768,"thread_start_us":10653,"threads_started":1}
I20260808 11:38:20.584844 4204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:20.523379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:42897 (local address 127.24.76.67:55379)
0808 11:38:20.524289 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:20.524332 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:20.526861 (+ 2529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:20.535950 (+ 9089us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:20.535958 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:20.536476 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:20.537066 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:20.537086 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.538652 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.538658 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:20.544141 (+ 5483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:20.544157 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.545388 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.545396 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:20.545657 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:20.546496 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:20.546530 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:20.552109 (+ 5579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:20.574947 (+ 22838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:20.574952 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:20.574956 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:20.577586 (+ 2630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:20.578153 (+ 567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:20.578157 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:20.578160 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:20.578329 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:20.579082 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:20.579087 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:20.579252 (+ 165us) client_negotiation.cc:769] Sending connection context
0808 11:38:20.579396 (+ 144us) client_negotiation.cc:240] Negotiation successful
0808 11:38:20.584421 (+ 5025us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":1655,"threads_started":1}
I20260808 11:38:20.586400 4189 heartbeater.cc:511] Master 127.24.76.124:45267 requested a full tablet report, sending...
I20260808 11:38:20.587421 4196 heartbeater.cc:347] Connected to a master server at 127.24.76.125:42897
I20260808 11:38:20.587571 4196 heartbeater.cc:464] Registering TS with master...
I20260808 11:38:20.588027 4206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:20.523393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.24.76.67:60349)
0808 11:38:20.539741 (+ 16348us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:20.539756 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:20.539909 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:20.540463 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:20.540468 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:20.541113 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:20.541486 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:20.541503 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.543099 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.543103 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:20.547188 (+ 4085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:20.547204 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.548361 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.548368 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:20.548582 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:20.550840 (+ 2258us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:20.550864 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:20.559668 (+ 8804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:20.568345 (+ 8677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:20.568350 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:20.568354 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:20.570445 (+ 2091us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:20.571087 (+ 642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:20.571092 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:20.571096 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:20.573421 (+ 2325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:20.575937 (+ 2516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:20.575946 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:20.576453 (+ 507us) client_negotiation.cc:769] Sending connection context
0808 11:38:20.583280 (+ 6827us) client_negotiation.cc:240] Negotiation successful
0808 11:38:20.585512 (+ 2232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16230,"mutex_wait_us":4707,"thread_start_us":16643,"threads_started":1}
I20260808 11:38:20.588075 4196 heartbeater.cc:511] Master 127.24.76.125:42897 requested a full tablet report, sending...
I20260808 11:38:20.588469 3474 ts_manager.cc:194] Registered new tserver with Master: c4d17c1c8b43495f81c16293bc8986f0 (127.24.76.67:43707)
I20260808 11:38:20.589252 4190 heartbeater.cc:347] Connected to a master server at 127.24.76.126:44823
I20260808 11:38:20.589388 4190 heartbeater.cc:464] Registering TS with master...
I20260808 11:38:20.589501 3181 ts_manager.cc:194] Registered new tserver with Master: c4d17c1c8b43495f81c16293bc8986f0 (127.24.76.67:43707)
I20260808 11:38:20.589864 4190 heartbeater.cc:511] Master 127.24.76.126:44823 requested a full tablet report, sending...
I20260808 11:38:20.590095 4214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:20.540721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37020 (local address 127.24.76.126:44823)
0808 11:38:20.549656 (+ 8935us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:20.549662 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:20.558118 (+ 8456us) server_negotiation.cc:407] Connection header received
0808 11:38:20.558665 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:20.558671 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:20.558763 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:20.558840 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:20.562822 (+ 3982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.564385 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.566566 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.567311 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.567412 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:20.571960 (+ 4548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:20.571966 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:20.571972 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:20.572007 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:20.577172 (+ 5165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:20.583475 (+ 6303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:20.583480 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:20.583483 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:20.583629 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:20.585405 (+ 1776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:20.585409 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:20.585412 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:20.585764 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:20.586036 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:20.589385 (+ 3349us) server_negotiation.cc:299] Negotiation successful
0808 11:38:20.589675 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8813,"thread_start_us":5566,"threads_started":1}
I20260808 11:38:20.590806 2294 ts_manager.cc:194] Re-registered known tserver with Master: c4d17c1c8b43495f81c16293bc8986f0 (127.24.76.67:43707)
I20260808 11:38:20.591926 2294 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.76.67@KRBTEST.COM'} at 127.24.76.67:60349
I20260808 11:38:20.601418 4219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:20.546239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56074 (local address 127.24.76.125:42897)
0808 11:38:20.573139 (+ 26900us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:20.573146 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:20.577216 (+ 4070us) server_negotiation.cc:407] Connection header received
0808 11:38:20.577287 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:20.577292 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:20.577416 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:20.577524 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:20.580634 (+ 3110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.582074 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.586060 (+ 3986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.586791 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.586883 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:20.594491 (+ 7608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:20.594499 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:20.594505 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:20.594542 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:20.598084 (+ 3542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:20.599006 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:20.599012 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:20.599016 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:20.599198 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:20.599629 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:20.599634 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:20.599638 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:20.600051 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:20.600360 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:20.600747 (+ 387us) server_negotiation.cc:299] Negotiation successful
0808 11:38:20.600969 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26673,"thread_start_us":25564,"threads_started":1}
I20260808 11:38:20.603823 4221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:20.548832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39468 (local address 127.24.76.124:45267)
0808 11:38:20.578291 (+ 29459us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:20.578297 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:20.583250 (+ 4953us) server_negotiation.cc:407] Connection header received
0808 11:38:20.583375 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:20.583380 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:20.583461 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:20.583552 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:20.585927 (+ 2375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.587144 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.592129 (+ 4985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.592837 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.592908 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:20.596738 (+ 3830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:20.596743 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:20.596747 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:20.596777 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:20.600279 (+ 3502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:20.601164 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:20.601169 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:20.601172 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:20.601303 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:20.602147 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:20.602151 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:20.602153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:20.602396 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:20.602625 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:20.603126 (+ 501us) server_negotiation.cc:299] Negotiation successful
0808 11:38:20.603340 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29325,"thread_start_us":29519,"threads_started":1}
I20260808 11:38:20.597173 2293 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37020:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:38:20.607378 2293 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:38:20.638489 4237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:20.628925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:33613 (local address 127.0.0.1:34016)
0808 11:38:20.630574 (+ 1649us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:20.630593 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:20.630765 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:20.631826 (+ 1061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:20.631830 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:20.631857 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:38:20.632420 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:20.632432 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.634088 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.634093 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:20.636345 (+ 2252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:20.636359 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.637656 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.637663 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:20.637834 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:20.637870 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 11:38:20.638004 (+ 134us) client_negotiation.cc:240] Negotiation successful
0808 11:38:20.638131 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1443,"thread_start_us":1305,"threads_started":1}
I20260808 11:38:20.639370 4240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:20.629241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34016 (local address 127.24.76.65:33613)
0808 11:38:20.630012 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:20.630022 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:20.630867 (+ 845us) server_negotiation.cc:407] Connection header received
0808 11:38:20.631000 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:20.631009 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:20.631267 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:20.631562 (+ 295us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:38:20.632602 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.633925 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.636535 (+ 2610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.637447 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.637582 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:20.637806 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:20.638880 (+ 1074us) server_negotiation.cc:299] Negotiation successful
0808 11:38:20.639076 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":401,"threads_started":1}
I20260808 11:38:20.646598 3648 tablet_service.cc:1467] Processing CreateTablet for tablet 8b74512b94be4453b4b73fc42cf56822 (DEFAULT_TABLE table=test-table [id=020489cf9abb4ac585f9ff3718937705]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:38:20.648257 3648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b74512b94be4453b4b73fc42cf56822. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:20.650851 4237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:20.630501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:43707 (local address 127.0.0.1:58678)
0808 11:38:20.639759 (+ 9258us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:20.639776 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:20.639862 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:20.643762 (+ 3900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:20.643767 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:20.643794 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:38:20.644052 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:20.644065 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.646419 (+ 2354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.646424 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:20.648279 (+ 1855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:20.648292 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.649859 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.649865 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:20.650112 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:20.650148 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 11:38:20.650212 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 11:38:20.650360 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9173,"mutex_wait_us":2,"thread_start_us":11514,"threads_started":1}
I20260808 11:38:20.651835 4245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:20.630917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58678 (local address 127.24.76.67:43707)
0808 11:38:20.642867 (+ 11950us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:20.642874 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:20.643225 (+ 351us) server_negotiation.cc:407] Connection header received
0808 11:38:20.643327 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:20.643335 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:20.643538 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:20.643741 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:38:20.644287 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.646278 (+ 1991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.648501 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.649651 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.651133 (+ 1482us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:20.651296 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:20.651349 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 11:38:20.651512 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11767,"thread_start_us":11527,"threads_started":1}
I20260808 11:38:20.652853 4235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:20.627804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:44933 (local address 127.0.0.1:39194)
0808 11:38:20.629129 (+ 1325us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:20.629151 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:20.629276 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:20.644954 (+ 15678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:20.644959 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:20.644984 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:38:20.645373 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:20.645384 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.647074 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.647079 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:20.649788 (+ 2709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:20.649803 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.652182 (+ 2379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.652187 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:20.652338 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:20.652362 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 11:38:20.652422 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 11:38:20.652515 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":908,"mutex_wait_us":17,"thread_start_us":702,"threads_started":1}
I20260808 11:38:20.652937 4236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:20.627999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39194 (local address 127.24.76.66:44933)
0808 11:38:20.644382 (+ 16383us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:20.644390 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:20.644418 (+ 28us) server_negotiation.cc:407] Connection header received
0808 11:38:20.644511 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:20.644519 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:20.644738 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:20.644963 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:38:20.645574 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.646925 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.650049 (+ 3124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.651996 (+ 1947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.652131 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:20.652386 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:20.652463 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 11:38:20.652644 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16218,"thread_start_us":15166,"threads_started":1}
I20260808 11:38:20.656689 4093 tablet_service.cc:1467] Processing CreateTablet for tablet 8b74512b94be4453b4b73fc42cf56822 (DEFAULT_TABLE table=test-table [id=020489cf9abb4ac585f9ff3718937705]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:38:20.656899 3873 tablet_service.cc:1467] Processing CreateTablet for tablet 8b74512b94be4453b4b73fc42cf56822 (DEFAULT_TABLE table=test-table [id=020489cf9abb4ac585f9ff3718937705]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:38:20.658126 4093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b74512b94be4453b4b73fc42cf56822. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:20.658303 3873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b74512b94be4453b4b73fc42cf56822. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:20.665047 4246 tablet_bootstrap.cc:492] T 8b74512b94be4453b4b73fc42cf56822 P 900623cd4f4a40c99098e6dfbe9f684f: Bootstrap starting.
I20260808 11:38:20.668895 4246 tablet_bootstrap.cc:654] T 8b74512b94be4453b4b73fc42cf56822 P 900623cd4f4a40c99098e6dfbe9f684f: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:20.670296 4246 log.cc:824] T 8b74512b94be4453b4b73fc42cf56822 P 900623cd4f4a40c99098e6dfbe9f684f: Log is configured to *not* fsync() on all Append() calls
I20260808 11:38:20.673676 4250 tablet_bootstrap.cc:492] T 8b74512b94be4453b4b73fc42cf56822 P 9b038a3b88794c9e8a63d670257a770d: Bootstrap starting.
I20260808 11:38:20.674252 4246 tablet_bootstrap.cc:492] T 8b74512b94be4453b4b73fc42cf56822 P 900623cd4f4a40c99098e6dfbe9f684f: No bootstrap required, opened a new log
I20260808 11:38:20.674501 4246 ts_tablet_manager.cc:1397] T 8b74512b94be4453b4b73fc42cf56822 P 900623cd4f4a40c99098e6dfbe9f684f: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260808 11:38:20.677156 4250 tablet_bootstrap.cc:654] T 8b74512b94be4453b4b73fc42cf56822 P 9b038a3b88794c9e8a63d670257a770d: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:20.678717 4250 log.cc:824] T 8b74512b94be4453b4b73fc42cf56822 P 9b038a3b88794c9e8a63d670257a770d: Log is configured to *not* fsync() on all Append() calls
I20260808 11:38:20.681202 4250 tablet_bootstrap.cc:492] T 8b74512b94be4453b4b73fc42cf56822 P 9b038a3b88794c9e8a63d670257a770d: No bootstrap required, opened a new log
I20260808 11:38:20.681488 4250 ts_tablet_manager.cc:1397] T 8b74512b94be4453b4b73fc42cf56822 P 9b038a3b88794c9e8a63d670257a770d: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260808 11:38:20.681596 4249 tablet_bootstrap.cc:492] T 8b74512b94be4453b4b73fc42cf56822 P c4d17c1c8b43495f81c16293bc8986f0: Bootstrap starting.
I20260808 11:38:20.683581 4246 raft_consensus.cc:348] T 8b74512b94be4453b4b73fc42cf56822 P 900623cd4f4a40c99098e6dfbe9f684f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b038a3b88794c9e8a63d670257a770d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44933 } } peers { permanent_uuid: "900623cd4f4a40c99098e6dfbe9f684f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 33613 } } peers { permanent_uuid: "c4d17c1c8b43495f81c16293bc8986f0" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 43707 } }
I20260808 11:38:20.684031 4246 raft_consensus.cc:374] T 8b74512b94be4453b4b73fc42cf56822 P 900623cd4f4a40c99098e6dfbe9f684f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:20.684216 4246 raft_consensus.cc:729] T 8b74512b94be4453b4b73fc42cf56822 P 900623cd4f4a40c99098e6dfbe9f684f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 900623cd4f4a40c99098e6dfbe9f684f, State: Initialized, Role: FOLLOWER
I20260808 11:38:20.684856 4246 consensus_queue.cc:260] T 8b74512b94be4453b4b73fc42cf56822 P 900623cd4f4a40c99098e6dfbe9f684f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b038a3b88794c9e8a63d670257a770d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44933 } } peers { permanent_uuid: "900623cd4f4a40c99098e6dfbe9f684f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 33613 } } peers { permanent_uuid: "c4d17c1c8b43495f81c16293bc8986f0" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 43707 } }
I20260808 11:38:20.685931 4249 tablet_bootstrap.cc:654] T 8b74512b94be4453b4b73fc42cf56822 P c4d17c1c8b43495f81c16293bc8986f0: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:20.686795 4246 ts_tablet_manager.cc:1428] T 8b74512b94be4453b4b73fc42cf56822 P 900623cd4f4a40c99098e6dfbe9f684f: Time spent starting tablet: real 0.012s user 0.007s sys 0.005s
I20260808 11:38:20.687565 4249 log.cc:824] T 8b74512b94be4453b4b73fc42cf56822 P c4d17c1c8b43495f81c16293bc8986f0: Log is configured to *not* fsync() on all Append() calls
I20260808 11:38:20.692487 4250 raft_consensus.cc:348] T 8b74512b94be4453b4b73fc42cf56822 P 9b038a3b88794c9e8a63d670257a770d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b038a3b88794c9e8a63d670257a770d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44933 } } peers { permanent_uuid: "900623cd4f4a40c99098e6dfbe9f684f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 33613 } } peers { permanent_uuid: "c4d17c1c8b43495f81c16293bc8986f0" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 43707 } }
I20260808 11:38:20.693011 4250 raft_consensus.cc:374] T 8b74512b94be4453b4b73fc42cf56822 P 9b038a3b88794c9e8a63d670257a770d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:20.693168 4250 raft_consensus.cc:729] T 8b74512b94be4453b4b73fc42cf56822 P 9b038a3b88794c9e8a63d670257a770d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b038a3b88794c9e8a63d670257a770d, State: Initialized, Role: FOLLOWER
I20260808 11:38:20.693204 4249 tablet_bootstrap.cc:492] T 8b74512b94be4453b4b73fc42cf56822 P c4d17c1c8b43495f81c16293bc8986f0: No bootstrap required, opened a new log
I20260808 11:38:20.693557 4249 ts_tablet_manager.cc:1397] T 8b74512b94be4453b4b73fc42cf56822 P c4d17c1c8b43495f81c16293bc8986f0: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.006s
I20260808 11:38:20.693979 4250 consensus_queue.cc:260] T 8b74512b94be4453b4b73fc42cf56822 P 9b038a3b88794c9e8a63d670257a770d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b038a3b88794c9e8a63d670257a770d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44933 } } peers { permanent_uuid: "900623cd4f4a40c99098e6dfbe9f684f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 33613 } } peers { permanent_uuid: "c4d17c1c8b43495f81c16293bc8986f0" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 43707 } }
I20260808 11:38:20.695411 3977 heartbeater.cc:503] Master 127.24.76.126:44823 was elected leader, sending a full tablet report...
I20260808 11:38:20.697002 4250 ts_tablet_manager.cc:1428] T 8b74512b94be4453b4b73fc42cf56822 P 9b038a3b88794c9e8a63d670257a770d: Time spent starting tablet: real 0.015s user 0.008s sys 0.008s
I20260808 11:38:20.703629 4249 raft_consensus.cc:348] T 8b74512b94be4453b4b73fc42cf56822 P c4d17c1c8b43495f81c16293bc8986f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b038a3b88794c9e8a63d670257a770d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44933 } } peers { permanent_uuid: "900623cd4f4a40c99098e6dfbe9f684f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 33613 } } peers { permanent_uuid: "c4d17c1c8b43495f81c16293bc8986f0" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 43707 } }
I20260808 11:38:20.703929 4249 raft_consensus.cc:374] T 8b74512b94be4453b4b73fc42cf56822 P c4d17c1c8b43495f81c16293bc8986f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:20.704069 4249 raft_consensus.cc:729] T 8b74512b94be4453b4b73fc42cf56822 P c4d17c1c8b43495f81c16293bc8986f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d17c1c8b43495f81c16293bc8986f0, State: Initialized, Role: FOLLOWER
I20260808 11:38:20.704558 4249 consensus_queue.cc:260] T 8b74512b94be4453b4b73fc42cf56822 P c4d17c1c8b43495f81c16293bc8986f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b038a3b88794c9e8a63d670257a770d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44933 } } peers { permanent_uuid: "900623cd4f4a40c99098e6dfbe9f684f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 33613 } } peers { permanent_uuid: "c4d17c1c8b43495f81c16293bc8986f0" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 43707 } }
I20260808 11:38:20.705696 4190 heartbeater.cc:503] Master 127.24.76.126:44823 was elected leader, sending a full tablet report...
I20260808 11:38:20.706218 4249 ts_tablet_manager.cc:1428] T 8b74512b94be4453b4b73fc42cf56822 P c4d17c1c8b43495f81c16293bc8986f0: Time spent starting tablet: real 0.013s user 0.004s sys 0.008s
W20260808 11:38:20.774179 4197 tablet.cc:2369] T 8b74512b94be4453b4b73fc42cf56822 P c4d17c1c8b43495f81c16293bc8986f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:38:20.778047 3754 tablet.cc:2369] T 8b74512b94be4453b4b73fc42cf56822 P 900623cd4f4a40c99098e6dfbe9f684f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:38:20.890986 3979 tablet.cc:2369] T 8b74512b94be4453b4b73fc42cf56822 P 9b038a3b88794c9e8a63d670257a770d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:38:20.907543 4256 raft_consensus.cc:482] T 8b74512b94be4453b4b73fc42cf56822 P c4d17c1c8b43495f81c16293bc8986f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:38:20.907894 4256 raft_consensus.cc:504] T 8b74512b94be4453b4b73fc42cf56822 P c4d17c1c8b43495f81c16293bc8986f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b038a3b88794c9e8a63d670257a770d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44933 } } peers { permanent_uuid: "900623cd4f4a40c99098e6dfbe9f684f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 33613 } } peers { permanent_uuid: "c4d17c1c8b43495f81c16293bc8986f0" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 43707 } }
I20260808 11:38:20.909714 4256 leader_election.cc:288] T 8b74512b94be4453b4b73fc42cf56822 P c4d17c1c8b43495f81c16293bc8986f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b038a3b88794c9e8a63d670257a770d (127.24.76.66:44933), 900623cd4f4a40c99098e6dfbe9f684f (127.24.76.65:33613)
I20260808 11:38:20.915701 4240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:20.910296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:39877 (local address 127.24.76.65:33613)
0808 11:38:20.910628 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:20.910634 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:20.910651 (+ 17us) server_negotiation.cc:407] Connection header received
0808 11:38:20.910711 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:20.910714 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:20.910781 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:20.910858 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:38:20.911225 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.912000 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.913976 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.914831 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.914975 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:20.915051 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:20.915303 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 11:38:20.915431 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260808 11:38:20.915841 4204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:20.909977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:33613 (local address 127.24.76.67:39877)
0808 11:38:20.910347 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:20.910368 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:20.910521 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:20.910894 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:20.910898 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:20.910922 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:38:20.911087 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:20.911096 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.912131 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.912135 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:20.913802 (+ 1667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:20.913823 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.914973 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.914978 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:20.915152 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:20.915179 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 11:38:20.915261 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 11:38:20.915394 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":30,"spinlock_wait_cycles":22784}
I20260808 11:38:20.915992 4236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:20.910224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:47539 (local address 127.24.76.66:44933)
0808 11:38:20.910442 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:20.910447 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:20.910467 (+ 20us) server_negotiation.cc:407] Connection header received
0808 11:38:20.910656 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:20.910661 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:20.910719 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:20.910794 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:38:20.911234 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.912042 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.913961 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.915017 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.915157 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:20.915229 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:20.915584 (+ 355us) server_negotiation.cc:299] Negotiation successful
0808 11:38:20.915702 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 11:38:20.916145 4206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:20.909977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:44933 (local address 127.24.76.67:47539)
0808 11:38:20.910320 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:20.910349 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:20.910510 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:20.910821 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:20.910826 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:20.910851 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:38:20.911087 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:20.911097 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.912183 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.912187 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:20.913803 (+ 1616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:20.913816 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:20.915226 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:20.915233 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:20.915437 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:20.915461 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 11:38:20.915539 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 11:38:20.915679 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":67,"spinlock_wait_cycles":41216}
I20260808 11:38:20.917043 3904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b74512b94be4453b4b73fc42cf56822" candidate_uuid: "c4d17c1c8b43495f81c16293bc8986f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b038a3b88794c9e8a63d670257a770d" is_pre_election: true
I20260808 11:38:20.917038 3685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b74512b94be4453b4b73fc42cf56822" candidate_uuid: "c4d17c1c8b43495f81c16293bc8986f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "900623cd4f4a40c99098e6dfbe9f684f" is_pre_election: true
I20260808 11:38:20.917536 3904 raft_consensus.cc:2454] T 8b74512b94be4453b4b73fc42cf56822 P 9b038a3b88794c9e8a63d670257a770d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d17c1c8b43495f81c16293bc8986f0 in term 0.
I20260808 11:38:20.917565 3685 raft_consensus.cc:2454] T 8b74512b94be4453b4b73fc42cf56822 P 900623cd4f4a40c99098e6dfbe9f684f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4d17c1c8b43495f81c16293bc8986f0 in term 0.
I20260808 11:38:20.918399 4025 leader_election.cc:302] T 8b74512b94be4453b4b73fc42cf56822 P c4d17c1c8b43495f81c16293bc8986f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 900623cd4f4a40c99098e6dfbe9f684f, c4d17c1c8b43495f81c16293bc8986f0; no voters:
I20260808 11:38:20.918831 4256 raft_consensus.cc:2781] T 8b74512b94be4453b4b73fc42cf56822 P c4d17c1c8b43495f81c16293bc8986f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:38:20.919016 4256 raft_consensus.cc:482] T 8b74512b94be4453b4b73fc42cf56822 P c4d17c1c8b43495f81c16293bc8986f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:38:20.919147 4256 raft_consensus.cc:3037] T 8b74512b94be4453b4b73fc42cf56822 P c4d17c1c8b43495f81c16293bc8986f0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:20.922293 4256 raft_consensus.cc:504] T 8b74512b94be4453b4b73fc42cf56822 P c4d17c1c8b43495f81c16293bc8986f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b038a3b88794c9e8a63d670257a770d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44933 } } peers { permanent_uuid: "900623cd4f4a40c99098e6dfbe9f684f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 33613 } } peers { permanent_uuid: "c4d17c1c8b43495f81c16293bc8986f0" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 43707 } }
I20260808 11:38:20.923017 4256 leader_election.cc:288] T 8b74512b94be4453b4b73fc42cf56822 P c4d17c1c8b43495f81c16293bc8986f0 [CANDIDATE]: Term 1 election: Requested vote from peers 9b038a3b88794c9e8a63d670257a770d (127.24.76.66:44933), 900623cd4f4a40c99098e6dfbe9f684f (127.24.76.65:33613)
I20260808 11:38:20.923527 3904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b74512b94be4453b4b73fc42cf56822" candidate_uuid: "c4d17c1c8b43495f81c16293bc8986f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b038a3b88794c9e8a63d670257a770d"
I20260808 11:38:20.923554 3685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b74512b94be4453b4b73fc42cf56822" candidate_uuid: "c4d17c1c8b43495f81c16293bc8986f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "900623cd4f4a40c99098e6dfbe9f684f"
I20260808 11:38:20.923704 3904 raft_consensus.cc:3037] T 8b74512b94be4453b4b73fc42cf56822 P 9b038a3b88794c9e8a63d670257a770d [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:20.923779 3685 raft_consensus.cc:3037] T 8b74512b94be4453b4b73fc42cf56822 P 900623cd4f4a40c99098e6dfbe9f684f [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:20.926456 3904 raft_consensus.cc:2454] T 8b74512b94be4453b4b73fc42cf56822 P 9b038a3b88794c9e8a63d670257a770d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d17c1c8b43495f81c16293bc8986f0 in term 1.
I20260808 11:38:20.926875 4024 leader_election.cc:302] T 8b74512b94be4453b4b73fc42cf56822 P c4d17c1c8b43495f81c16293bc8986f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b038a3b88794c9e8a63d670257a770d, c4d17c1c8b43495f81c16293bc8986f0; no voters:
I20260808 11:38:20.927278 4256 raft_consensus.cc:2781] T 8b74512b94be4453b4b73fc42cf56822 P c4d17c1c8b43495f81c16293bc8986f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:38:20.927632 3685 raft_consensus.cc:2454] T 8b74512b94be4453b4b73fc42cf56822 P 900623cd4f4a40c99098e6dfbe9f684f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4d17c1c8b43495f81c16293bc8986f0 in term 1.
I20260808 11:38:20.928464 4256 raft_consensus.cc:686] T 8b74512b94be4453b4b73fc42cf56822 P c4d17c1c8b43495f81c16293bc8986f0 [term 1 LEADER]: Becoming Leader. State: Replica: c4d17c1c8b43495f81c16293bc8986f0, State: Running, Role: LEADER
I20260808 11:38:20.929100 4256 consensus_queue.cc:237] T 8b74512b94be4453b4b73fc42cf56822 P c4d17c1c8b43495f81c16293bc8986f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b038a3b88794c9e8a63d670257a770d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44933 } } peers { permanent_uuid: "900623cd4f4a40c99098e6dfbe9f684f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 33613 } } peers { permanent_uuid: "c4d17c1c8b43495f81c16293bc8986f0" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 43707 } }
I20260808 11:38:20.935550 2293 catalog_manager.cc:5495] T 8b74512b94be4453b4b73fc42cf56822 P c4d17c1c8b43495f81c16293bc8986f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4d17c1c8b43495f81c16293bc8986f0 (127.24.76.67). New cstate: current_term: 1 leader_uuid: "c4d17c1c8b43495f81c16293bc8986f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b038a3b88794c9e8a63d670257a770d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "900623cd4f4a40c99098e6dfbe9f684f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 33613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4d17c1c8b43495f81c16293bc8986f0" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 43707 } health_report { overall_health: HEALTHY } } }
I20260808 11:38:21.005257 4245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:20.999542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58680 (local address 127.24.76.67:43707)
0808 11:38:20.999693 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:20.999697 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:20.999715 (+ 18us) server_negotiation.cc:407] Connection header received
0808 11:38:20.999901 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:20.999905 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:20.999987 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:21.000123 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:38:21.000493 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:21.001379 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:21.002895 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:21.003555 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:21.003658 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:21.004235 (+ 577us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:38:21.004719 (+ 484us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:38:21.004829 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:21.004943 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 11:38:21.005022 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 11:38:21.014197 3685 raft_consensus.cc:1261] T 8b74512b94be4453b4b73fc42cf56822 P 900623cd4f4a40c99098e6dfbe9f684f [term 1 FOLLOWER]: Refusing update from remote peer c4d17c1c8b43495f81c16293bc8986f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:38:21.014272 3904 raft_consensus.cc:1261] T 8b74512b94be4453b4b73fc42cf56822 P 9b038a3b88794c9e8a63d670257a770d [term 1 FOLLOWER]: Refusing update from remote peer c4d17c1c8b43495f81c16293bc8986f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:38:21.015198 4257 consensus_queue.cc:1035] T 8b74512b94be4453b4b73fc42cf56822 P c4d17c1c8b43495f81c16293bc8986f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b038a3b88794c9e8a63d670257a770d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:38:21.015643 4256 consensus_queue.cc:1035] T 8b74512b94be4453b4b73fc42cf56822 P c4d17c1c8b43495f81c16293bc8986f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "900623cd4f4a40c99098e6dfbe9f684f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 33613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:38:21.031832 4264 mvcc.cc:205] Tried to move back new op lower bound from 7316230557744107520 to 7316230557409996800. Current Snapshot: MvccSnapshot[applied={T|T < 7316230557744107520}]
I20260808 11:38:21.035917 4263 mvcc.cc:205] Tried to move back new op lower bound from 7316230557744107520 to 7316230557409996800. Current Snapshot: MvccSnapshot[applied={T|T < 7316230557744107520 or (T in {7316230557744107520})}]
I20260808 11:38:21.039382 4265 mvcc.cc:205] Tried to move back new op lower bound from 7316230557744107520 to 7316230557409996800. Current Snapshot: MvccSnapshot[applied={T|T < 7316230557744107520}]
I20260808 11:38:21.050362 2293 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:38:21.050628 2293 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:38:21.057317 2293 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6ca9db23e8f945df88bffa6f2836ea9a: Sending DeleteTablet for 3 replicas of tablet 8b74512b94be4453b4b73fc42cf56822
I20260808 11:38:21.058573 4093 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b74512b94be4453b4b73fc42cf56822 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:38:21 UTC) from {username='oryx', principal='oryx/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:58678
I20260808 11:38:21.058562 3873 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b74512b94be4453b4b73fc42cf56822 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:38:21 UTC) from {username='oryx', principal='oryx/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:39194
I20260808 11:38:21.058537 3648 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b74512b94be4453b4b73fc42cf56822 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:38:21 UTC) from {username='oryx', principal='oryx/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:34016
I20260808 11:38:21.059535 4279 tablet_replica.cc:330] T 8b74512b94be4453b4b73fc42cf56822 P 900623cd4f4a40c99098e6dfbe9f684f: stopping tablet replica
I20260808 11:38:21.060003 4279 raft_consensus.cc:2229] T 8b74512b94be4453b4b73fc42cf56822 P 900623cd4f4a40c99098e6dfbe9f684f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:38:21.060549 4279 raft_consensus.cc:2258] T 8b74512b94be4453b4b73fc42cf56822 P 900623cd4f4a40c99098e6dfbe9f684f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:38:21.062887 4279 ts_tablet_manager.cc:1905] T 8b74512b94be4453b4b73fc42cf56822 P 900623cd4f4a40c99098e6dfbe9f684f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:38:21.071852 4279 ts_tablet_manager.cc:1918] T 8b74512b94be4453b4b73fc42cf56822 P 900623cd4f4a40c99098e6dfbe9f684f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:38:21.072077 4279 log.cc:1196] T 8b74512b94be4453b4b73fc42cf56822 P 900623cd4f4a40c99098e6dfbe9f684f: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-0/wal/wals/8b74512b94be4453b4b73fc42cf56822
I20260808 11:38:21.072834 4279 ts_tablet_manager.cc:1939] T 8b74512b94be4453b4b73fc42cf56822 P 900623cd4f4a40c99098e6dfbe9f684f: Deleting consensus metadata
I20260808 11:38:21.075003 2267 catalog_manager.cc:4841] TS 900623cd4f4a40c99098e6dfbe9f684f (127.24.76.65:33613): tablet 8b74512b94be4453b4b73fc42cf56822 (table test-table [id=020489cf9abb4ac585f9ff3718937705]) successfully deleted
I20260808 11:38:21.079559 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 2241
I20260808 11:38:21.080111 4277 tablet_replica.cc:330] T 8b74512b94be4453b4b73fc42cf56822 P 9b038a3b88794c9e8a63d670257a770d: stopping tablet replica
I20260808 11:38:21.080642 4277 raft_consensus.cc:2229] T 8b74512b94be4453b4b73fc42cf56822 P 9b038a3b88794c9e8a63d670257a770d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:38:21.080921 2339 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:21.081252 4277 raft_consensus.cc:2258] T 8b74512b94be4453b4b73fc42cf56822 P 9b038a3b88794c9e8a63d670257a770d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:38:21.085670 4278 tablet_replica.cc:330] T 8b74512b94be4453b4b73fc42cf56822 P c4d17c1c8b43495f81c16293bc8986f0: stopping tablet replica
I20260808 11:38:21.088317 4277 ts_tablet_manager.cc:1905] T 8b74512b94be4453b4b73fc42cf56822 P 9b038a3b88794c9e8a63d670257a770d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:38:21.086102 4278 raft_consensus.cc:2229] T 8b74512b94be4453b4b73fc42cf56822 P c4d17c1c8b43495f81c16293bc8986f0 [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:38:21.095818 4278 raft_consensus.cc:2258] T 8b74512b94be4453b4b73fc42cf56822 P c4d17c1c8b43495f81c16293bc8986f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:38:21.096226 4277 ts_tablet_manager.cc:1918] T 8b74512b94be4453b4b73fc42cf56822 P 9b038a3b88794c9e8a63d670257a770d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:38:21.096499 4277 log.cc:1196] T 8b74512b94be4453b4b73fc42cf56822 P 9b038a3b88794c9e8a63d670257a770d: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-1/wal/wals/8b74512b94be4453b4b73fc42cf56822
I20260808 11:38:21.097256 4277 ts_tablet_manager.cc:1939] T 8b74512b94be4453b4b73fc42cf56822 P 9b038a3b88794c9e8a63d670257a770d: Deleting consensus metadata
I20260808 11:38:21.098767 4278 ts_tablet_manager.cc:1905] T 8b74512b94be4453b4b73fc42cf56822 P c4d17c1c8b43495f81c16293bc8986f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:38:21.107044 4278 ts_tablet_manager.cc:1918] T 8b74512b94be4453b4b73fc42cf56822 P c4d17c1c8b43495f81c16293bc8986f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:38:21.107293 4278 log.cc:1196] T 8b74512b94be4453b4b73fc42cf56822 P c4d17c1c8b43495f81c16293bc8986f0: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-2/wal/wals/8b74512b94be4453b4b73fc42cf56822
I20260808 11:38:21.107982 4278 ts_tablet_manager.cc:1939] T 8b74512b94be4453b4b73fc42cf56822 P c4d17c1c8b43495f81c16293bc8986f0: Deleting consensus metadata
I20260808 11:38:21.109755 4219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:21.063575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56078 (local address 127.24.76.125:42897)
0808 11:38:21.063759 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:21.063766 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:21.079097 (+ 15331us) server_negotiation.cc:407] Connection header received
0808 11:38:21.079352 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:21.079358 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:21.079443 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:21.079560 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:21.084958 (+ 5398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:21.086128 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:21.095020 (+ 8892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:21.095757 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:21.095894 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:21.101978 (+ 6084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:21.101985 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:21.101990 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:21.102032 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:21.106111 (+ 4079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:21.107018 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:21.107024 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:21.107028 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:21.107209 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:21.107687 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:21.107692 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:21.107695 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:21.108123 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:21.108454 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:21.109127 (+ 673us) server_negotiation.cc:299] Negotiation successful
0808 11:38:21.109366 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 11:38:21.127861 4221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:21.078849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39472 (local address 127.24.76.124:45267)
0808 11:38:21.079044 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:21.079051 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:21.110791 (+ 31740us) server_negotiation.cc:407] Connection header received
0808 11:38:21.110952 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:21.110960 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:21.111060 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:21.111174 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:21.112784 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:21.113909 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:21.115992 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:21.116769 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:21.116933 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:21.120720 (+ 3787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:21.120726 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:21.120731 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:21.120772 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:21.124558 (+ 3786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:21.125477 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:21.125483 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:21.125487 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:21.125681 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:21.126194 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:21.126200 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:21.126204 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:21.126627 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:21.126931 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:21.127202 (+ 271us) server_negotiation.cc:299] Negotiation successful
0808 11:38:21.127518 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 11:38:21.218518 2267 catalog_manager.cc:4841] TS c4d17c1c8b43495f81c16293bc8986f0 (127.24.76.67:43707): tablet 8b74512b94be4453b4b73fc42cf56822 (table test-table [id=020489cf9abb4ac585f9ff3718937705]) successfully deleted
I20260808 11:38:21.219573 2265 catalog_manager.cc:4841] TS 9b038a3b88794c9e8a63d670257a770d (127.24.76.66:44933): tablet 8b74512b94be4453b4b73fc42cf56822 (table test-table [id=020489cf9abb4ac585f9ff3718937705]) successfully deleted
I20260808 11:38:21.220242 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 2241
I20260808 11:38:21.281189 4219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:21.256820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56092 (local address 127.24.76.125:42897)
0808 11:38:21.257003 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:21.257009 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:21.263797 (+ 6788us) server_negotiation.cc:407] Connection header received
0808 11:38:21.264966 (+ 1169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:21.264972 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:21.265067 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:21.265172 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:21.266689 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:21.267595 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:21.269475 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:21.269988 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:21.270122 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:21.274210 (+ 4088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:21.274215 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:21.274220 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:21.274258 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:21.278096 (+ 3838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:21.278927 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:21.278933 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:21.278937 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:21.279089 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:21.279471 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:21.279476 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:21.279479 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:21.279795 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:21.280018 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:21.280562 (+ 544us) server_negotiation.cc:299] Negotiation successful
0808 11:38:21.280772 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 11:38:21.281620 4221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:21.256477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39486 (local address 127.24.76.124:45267)
0808 11:38:21.256682 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:21.256689 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:21.264527 (+ 7838us) server_negotiation.cc:407] Connection header received
0808 11:38:21.264989 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:21.264995 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:21.265092 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:21.265162 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:21.267366 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:21.268197 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:21.269579 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:21.270257 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:21.270450 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:21.274947 (+ 4497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:21.274954 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:21.274958 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:21.274998 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:21.278289 (+ 3291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:21.279141 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:21.279146 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:21.279149 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:21.279306 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:21.279753 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:21.279758 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:21.279762 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:21.280105 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:21.280337 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:21.280932 (+ 595us) server_negotiation.cc:299] Negotiation successful
0808 11:38:21.281176 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 11:38:22.038219 4292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.037504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.24.76.65:57445)
0808 11:38:22.037931 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:22.038057 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":195,"threads_started":1}
W20260808 11:38:22.038679 3751 heartbeater.cc:644] Failed to heartbeat to 127.24.76.126:44823 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.76.126:44823: connect: Connection refused (error 111)
I20260808 11:38:22.041545 4294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.040763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.24.76.67:51079)
0808 11:38:22.041149 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:22.041315 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":192,"threads_started":1}
W20260808 11:38:22.042042 4190 heartbeater.cc:644] Failed to heartbeat to 127.24.76.126:44823 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.76.126:44823: connect: Connection refused (error 111)
I20260808 11:38:22.046061 4297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.045314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.24.76.66:49173)
0808 11:38:22.045745 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:22.045860 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":196,"threads_started":1}
W20260808 11:38:22.046402 3977 heartbeater.cc:644] Failed to heartbeat to 127.24.76.126:44823 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.76.126:44823: connect: Connection refused (error 111)
I20260808 11:38:22.557420 4298 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ca9db23e8f945df88bffa6f2836ea9a)
I20260808 11:38:22.557683 4298 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: false } } peers { permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 45267 } attrs { promote: false } }
I20260808 11:38:22.558810 4299 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ca9db23e8f945df88bffa6f2836ea9a)
I20260808 11:38:22.559060 4299 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: false } } peers { permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 45267 } attrs { promote: false } }
I20260808 11:38:22.559206 4298 leader_election.cc:288] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ca9db23e8f945df88bffa6f2836ea9a (127.24.76.126:44823), f7c33bdb0de24324977cff0e51bf1d47 (127.24.76.124:45267)
I20260808 11:38:22.566627 4299 leader_election.cc:288] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ca9db23e8f945df88bffa6f2836ea9a (127.24.76.126:44823), 79a1352c611a496ba0577480a04e026f (127.24.76.125:42897)
I20260808 11:38:22.569733 4303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.559302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.0.0.1:37458)
0808 11:38:22.569020 (+ 9718us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:22.569226 (+ 206us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":9565,"thread_start_us":10024,"threads_started":1}
W20260808 11:38:22.572510 3162 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.76.126:44823: connect: Connection refused (error 111)
I20260808 11:38:22.573786 4305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.560645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.0.0.1:37460)
0808 11:38:22.573288 (+ 12643us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:22.573507 (+ 219us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12504,"threads_started":1}
W20260808 11:38:22.574290 3440 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.76.126:44823: connect: Connection refused (error 111)
I20260808 11:38:22.577832 4301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.559383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:45267 (local address 127.0.0.1:39498)
0808 11:38:22.569352 (+ 9969us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:22.569372 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:22.569527 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:22.570649 (+ 1122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:22.570655 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:22.570684 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:38:22.571045 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:22.571059 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:22.572784 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:22.572790 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:22.575015 (+ 2225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:22.575033 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:22.576954 (+ 1921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:22.576961 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:22.577188 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:22.577227 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 11:38:22.577315 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 11:38:22.577463 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9859,"thread_start_us":9376,"threads_started":1}
I20260808 11:38:22.578591 4304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.559772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39498 (local address 127.24.76.124:45267)
0808 11:38:22.570166 (+ 10394us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:22.570172 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:22.570189 (+ 17us) server_negotiation.cc:407] Connection header received
0808 11:38:22.570254 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:22.570259 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:22.570335 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:22.570434 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:38:22.571303 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:22.572597 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:22.575210 (+ 2613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:22.576628 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:22.576751 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:22.576859 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:22.578051 (+ 1192us) server_negotiation.cc:299] Negotiation successful
0808 11:38:22.578208 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10265,"thread_start_us":9984,"threads_started":1}
I20260808 11:38:22.580687 3486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "79a1352c611a496ba0577480a04e026f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f7c33bdb0de24324977cff0e51bf1d47" is_pre_election: true
I20260808 11:38:22.581082 3486 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79a1352c611a496ba0577480a04e026f in term 1.
I20260808 11:38:22.581418 4303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.581006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.0.0.1:37466)
0808 11:38:22.581209 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:22.581298 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260808 11:38:22.581672 3164 leader_election.cc:302] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79a1352c611a496ba0577480a04e026f, f7c33bdb0de24324977cff0e51bf1d47; no voters:
I20260808 11:38:22.582260 4298 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 11:38:22.582577 4307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.582112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.0.0.1:37474)
0808 11:38:22.582246 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:22.582393 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 11:38:22.582618 4298 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6ca9db23e8f945df88bffa6f2836ea9a)
W20260808 11:38:22.582886 3440 leader_election.cc:334] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6ca9db23e8f945df88bffa6f2836ea9a (127.24.76.126:44823): Network error: Client connection negotiation failed: client connection to 127.24.76.126:44823: connect: Connection refused (error 111)
I20260808 11:38:22.583029 4298 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [term 1 FOLLOWER]: Advancing to term 2
I20260808 11:38:22.585462 4298 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: false } } peers { permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 45267 } attrs { promote: false } }
W20260808 11:38:22.585742 3162 leader_election.cc:334] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6ca9db23e8f945df88bffa6f2836ea9a (127.24.76.126:44823): Network error: Client connection negotiation failed: client connection to 127.24.76.126:44823: connect: Connection refused (error 111)
I20260808 11:38:22.586522 4298 leader_election.cc:288] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [CANDIDATE]: Term 2 election: Requested vote from peers 6ca9db23e8f945df88bffa6f2836ea9a (127.24.76.126:44823), f7c33bdb0de24324977cff0e51bf1d47 (127.24.76.124:45267)
I20260808 11:38:22.586958 4303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.586433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.0.0.1:37484)
0808 11:38:22.586721 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:22.586814 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
I20260808 11:38:22.587011 3486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "79a1352c611a496ba0577480a04e026f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f7c33bdb0de24324977cff0e51bf1d47"
I20260808 11:38:22.587229 3486 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [term 1 FOLLOWER]: Advancing to term 2
I20260808 11:38:22.588141 4303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.587692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.0.0.1:37488)
0808 11:38:22.587895 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:22.588006 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
W20260808 11:38:22.588389 3162 leader_election.cc:334] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 6ca9db23e8f945df88bffa6f2836ea9a (127.24.76.126:44823): Network error: Client connection negotiation failed: client connection to 127.24.76.126:44823: connect: Connection refused (error 111)
I20260808 11:38:22.590065 4305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.570166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:42897 (local address 127.0.0.1:56098)
0808 11:38:22.575091 (+ 4925us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:22.575109 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:22.575228 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:22.579761 (+ 4533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:22.579767 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:22.579805 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:38:22.580065 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:22.580078 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:22.585621 (+ 5543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:22.585626 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:22.587487 (+ 1861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:22.587503 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:22.589319 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:22.589325 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:22.589512 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:22.589546 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 11:38:22.589630 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 11:38:22.589767 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4836,"thread_start_us":9888,"threads_started":1}
I20260808 11:38:22.590317 3486 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79a1352c611a496ba0577480a04e026f in term 2.
I20260808 11:38:22.590674 4309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.570124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56098 (local address 127.24.76.125:42897)
0808 11:38:22.579482 (+ 9358us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:22.579489 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:22.579508 (+ 19us) server_negotiation.cc:407] Connection header received
0808 11:38:22.579567 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:22.579572 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:22.579642 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:22.579739 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:38:22.580334 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:22.585474 (+ 5140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:22.587721 (+ 2247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:22.588972 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:22.589088 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:22.589192 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:22.590256 (+ 1064us) server_negotiation.cc:299] Negotiation successful
0808 11:38:22.590394 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9225,"thread_start_us":9057,"threads_started":1}
I20260808 11:38:22.591005 3164 leader_election.cc:302] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 79a1352c611a496ba0577480a04e026f, f7c33bdb0de24324977cff0e51bf1d47; no voters: 6ca9db23e8f945df88bffa6f2836ea9a
I20260808 11:38:22.591496 3193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f7c33bdb0de24324977cff0e51bf1d47" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "79a1352c611a496ba0577480a04e026f" is_pre_election: true
I20260808 11:38:22.591773 3193 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f7c33bdb0de24324977cff0e51bf1d47 in current term 2: Already voted for candidate 79a1352c611a496ba0577480a04e026f in this term.
I20260808 11:38:22.592089 4298 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [term 2 FOLLOWER]: Leader election won for term 2
I20260808 11:38:22.592310 3441 leader_election.cc:302] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f7c33bdb0de24324977cff0e51bf1d47; no voters: 6ca9db23e8f945df88bffa6f2836ea9a, 79a1352c611a496ba0577480a04e026f
I20260808 11:38:22.592612 4299 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 11:38:22.593017 4298 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [term 2 LEADER]: Becoming Leader. State: Replica: 79a1352c611a496ba0577480a04e026f, State: Running, Role: LEADER
I20260808 11:38:22.593436 4298 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: false } } peers { permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 45267 } attrs { promote: false } }
I20260808 11:38:22.595804 4316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79a1352c611a496ba0577480a04e026f. Latest consensus state: current_term: 2 leader_uuid: "79a1352c611a496ba0577480a04e026f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: false } } peers { permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 45267 } attrs { promote: false } } }
I20260808 11:38:22.596218 4316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [sys.catalog]: This master's current role is: LEADER
I20260808 11:38:22.596916 4318 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:38:22.600939 4318 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:38:22.601755 4318 catalog_manager.cc:1259] Loaded cluster ID: 76425cb76ac444de9d8e156385ebe8f6
I20260808 11:38:22.601831 4318 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:38:22.602965 4318 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:38:22.603750 4318 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f: Loaded TSK: 0
I20260808 11:38:22.604607 4318 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:38:22.654678 3181 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56092:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:38:22.661610 3181 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:38:22.665705 4303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.665335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.0.0.1:51442)
0808 11:38:22.665494 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:22.665567 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 11:38:22.665776 3486 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [term 2 FOLLOWER]: Refusing update from remote peer 79a1352c611a496ba0577480a04e026f: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260808 11:38:22.666719 4303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.666316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.0.0.1:51456)
0808 11:38:22.666488 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:22.666582 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 11:38:22.666563 4316 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 45267 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260808 11:38:22.666925 3162 consensus_peers.cc:487] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f -> Peer 6ca9db23e8f945df88bffa6f2836ea9a (127.24.76.126:44823): Couldn't send request to peer 6ca9db23e8f945df88bffa6f2836ea9a. Status: Network error: Client connection negotiation failed: client connection to 127.24.76.126:44823: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 11:38:22.669584 4299 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79a1352c611a496ba0577480a04e026f. Latest consensus state: current_term: 2 leader_uuid: "79a1352c611a496ba0577480a04e026f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: false } } peers { permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 45267 } attrs { promote: false } } }
I20260808 11:38:22.669831 4299 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:38:22.670725 4316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "79a1352c611a496ba0577480a04e026f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: false } } peers { permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 45267 } attrs { promote: false } } }
I20260808 11:38:22.670995 4316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f [sys.catalog]: This master's current role is: LEADER
I20260808 11:38:22.670997 4320 mvcc.cc:205] Tried to move back new op lower bound from 7316230564512481280 to 7316230564226097152. Current Snapshot: MvccSnapshot[applied={T|T < 7316230564512481280}]
I20260808 11:38:22.671705 4299 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "79a1352c611a496ba0577480a04e026f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6ca9db23e8f945df88bffa6f2836ea9a" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44823 } } peers { permanent_uuid: "79a1352c611a496ba0577480a04e026f" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 42897 } attrs { promote: false } } peers { permanent_uuid: "f7c33bdb0de24324977cff0e51bf1d47" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 45267 } attrs { promote: false } } }
I20260808 11:38:22.671901 4299 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7c33bdb0de24324977cff0e51bf1d47 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:38:22.695837 4301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.678910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:44933 (local address 127.0.0.1:54548)
0808 11:38:22.679054 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:22.679071 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:22.679156 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:22.690886 (+ 11730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:22.690891 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:22.690917 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:38:22.691132 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:22.691145 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:22.692257 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:22.692262 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:22.693792 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:22.693805 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:22.695138 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:22.695145 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:22.695305 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:22.695328 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 11:38:22.695424 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 11:38:22.695557 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 11:38:22.695854 4332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.679296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54548 (local address 127.24.76.66:44933)
0808 11:38:22.690326 (+ 11030us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:22.690333 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:22.690357 (+ 24us) server_negotiation.cc:407] Connection header received
0808 11:38:22.690438 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:22.690444 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:22.690736 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:22.690857 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:38:22.691313 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:22.692132 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:22.693958 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:22.694986 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:22.695090 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:22.695195 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:22.695461 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 11:38:22.695594 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10887,"thread_start_us":10783,"threads_started":1}
I20260808 11:38:22.697311 3873 tablet_service.cc:1467] Processing CreateTablet for tablet e760ac585f944970b77d88b082f507da (DEFAULT_TABLE table=test-table [id=2727f60fe07e42c5a23bda57e184d76f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:38:22.697408 4303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.678559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:43707 (local address 127.0.0.1:59488)
0808 11:38:22.678704 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:22.678722 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:22.678836 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:22.691625 (+ 12789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:22.691629 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:22.691654 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:38:22.691942 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:22.691959 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:22.693251 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:22.693257 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:22.695063 (+ 1806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:22.695078 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:22.696635 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:22.696641 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:22.696817 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:22.696857 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 11:38:22.696922 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 11:38:22.697089 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":28}
I20260808 11:38:22.697754 3873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e760ac585f944970b77d88b082f507da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:22.698217 4331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.678690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59488 (local address 127.24.76.67:43707)
0808 11:38:22.691315 (+ 12625us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:22.691321 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:22.691340 (+ 19us) server_negotiation.cc:407] Connection header received
0808 11:38:22.691406 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:22.691410 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:22.691497 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:22.691594 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:38:22.692081 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:22.693117 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:22.695249 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:22.696371 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:22.697615 (+ 1244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:22.697712 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:22.697765 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 11:38:22.697921 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12512,"thread_start_us":12419,"threads_started":1}
I20260808 11:38:22.698925 4093 tablet_service.cc:1467] Processing CreateTablet for tablet e760ac585f944970b77d88b082f507da (DEFAULT_TABLE table=test-table [id=2727f60fe07e42c5a23bda57e184d76f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:38:22.699298 4093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e760ac585f944970b77d88b082f507da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:22.700521 4335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.679461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59210 (local address 127.24.76.65:33613)
0808 11:38:22.693792 (+ 14331us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:22.693798 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:22.693816 (+ 18us) server_negotiation.cc:407] Connection header received
0808 11:38:22.693994 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:22.694001 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:22.694080 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:22.694186 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:38:22.694816 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:22.695916 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:22.698408 (+ 2492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:22.699537 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:22.699670 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:22.699771 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:22.700092 (+ 321us) server_negotiation.cc:299] Negotiation successful
0808 11:38:22.700232 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14210,"thread_start_us":14434,"threads_started":1}
I20260808 11:38:22.700543 4333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.679350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:33613 (local address 127.0.0.1:59210)
0808 11:38:22.689438 (+ 10088us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:22.689456 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:22.689576 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:22.694218 (+ 4642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:22.694223 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:22.694248 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:38:22.694662 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:22.694673 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:22.696085 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:22.696090 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:22.698149 (+ 2059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:22.698163 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:22.699758 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:22.699765 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:22.699928 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:22.699958 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 11:38:22.700052 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 11:38:22.700168 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9958,"mutex_wait_us":1,"thread_start_us":9769,"threads_started":1}
I20260808 11:38:22.703703 4336 tablet_bootstrap.cc:492] T e760ac585f944970b77d88b082f507da P 9b038a3b88794c9e8a63d670257a770d: Bootstrap starting.
I20260808 11:38:22.704991 4337 tablet_bootstrap.cc:492] T e760ac585f944970b77d88b082f507da P c4d17c1c8b43495f81c16293bc8986f0: Bootstrap starting.
I20260808 11:38:22.705324 3648 tablet_service.cc:1467] Processing CreateTablet for tablet e760ac585f944970b77d88b082f507da (DEFAULT_TABLE table=test-table [id=2727f60fe07e42c5a23bda57e184d76f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:38:22.705441 4336 tablet_bootstrap.cc:654] T e760ac585f944970b77d88b082f507da P 9b038a3b88794c9e8a63d670257a770d: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:22.705729 3648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e760ac585f944970b77d88b082f507da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:22.706734 4337 tablet_bootstrap.cc:654] T e760ac585f944970b77d88b082f507da P c4d17c1c8b43495f81c16293bc8986f0: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:22.707315 4336 tablet_bootstrap.cc:492] T e760ac585f944970b77d88b082f507da P 9b038a3b88794c9e8a63d670257a770d: No bootstrap required, opened a new log
I20260808 11:38:22.707419 4336 ts_tablet_manager.cc:1397] T e760ac585f944970b77d88b082f507da P 9b038a3b88794c9e8a63d670257a770d: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260808 11:38:22.708153 4336 raft_consensus.cc:348] T e760ac585f944970b77d88b082f507da P 9b038a3b88794c9e8a63d670257a770d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d17c1c8b43495f81c16293bc8986f0" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 43707 } } peers { permanent_uuid: "9b038a3b88794c9e8a63d670257a770d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44933 } } peers { permanent_uuid: "900623cd4f4a40c99098e6dfbe9f684f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 33613 } }
I20260808 11:38:22.708395 4336 raft_consensus.cc:374] T e760ac585f944970b77d88b082f507da P 9b038a3b88794c9e8a63d670257a770d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:22.708473 4336 raft_consensus.cc:729] T e760ac585f944970b77d88b082f507da P 9b038a3b88794c9e8a63d670257a770d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b038a3b88794c9e8a63d670257a770d, State: Initialized, Role: FOLLOWER
I20260808 11:38:22.708694 4336 consensus_queue.cc:260] T e760ac585f944970b77d88b082f507da P 9b038a3b88794c9e8a63d670257a770d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d17c1c8b43495f81c16293bc8986f0" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 43707 } } peers { permanent_uuid: "9b038a3b88794c9e8a63d670257a770d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44933 } } peers { permanent_uuid: "900623cd4f4a40c99098e6dfbe9f684f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 33613 } }
I20260808 11:38:22.709041 4337 tablet_bootstrap.cc:492] T e760ac585f944970b77d88b082f507da P c4d17c1c8b43495f81c16293bc8986f0: No bootstrap required, opened a new log
I20260808 11:38:22.709169 4337 ts_tablet_manager.cc:1397] T e760ac585f944970b77d88b082f507da P c4d17c1c8b43495f81c16293bc8986f0: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260808 11:38:22.709440 4336 ts_tablet_manager.cc:1428] T e760ac585f944970b77d88b082f507da P 9b038a3b88794c9e8a63d670257a770d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 11:38:22.709959 4337 raft_consensus.cc:348] T e760ac585f944970b77d88b082f507da P c4d17c1c8b43495f81c16293bc8986f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d17c1c8b43495f81c16293bc8986f0" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 43707 } } peers { permanent_uuid: "9b038a3b88794c9e8a63d670257a770d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44933 } } peers { permanent_uuid: "900623cd4f4a40c99098e6dfbe9f684f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 33613 } }
I20260808 11:38:22.710245 4337 raft_consensus.cc:374] T e760ac585f944970b77d88b082f507da P c4d17c1c8b43495f81c16293bc8986f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:22.710342 4337 raft_consensus.cc:729] T e760ac585f944970b77d88b082f507da P c4d17c1c8b43495f81c16293bc8986f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d17c1c8b43495f81c16293bc8986f0, State: Initialized, Role: FOLLOWER
I20260808 11:38:22.710592 4337 consensus_queue.cc:260] T e760ac585f944970b77d88b082f507da P c4d17c1c8b43495f81c16293bc8986f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d17c1c8b43495f81c16293bc8986f0" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 43707 } } peers { permanent_uuid: "9b038a3b88794c9e8a63d670257a770d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44933 } } peers { permanent_uuid: "900623cd4f4a40c99098e6dfbe9f684f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 33613 } }
I20260808 11:38:22.716537 3978 heartbeater.cc:503] Master 127.24.76.125:42897 was elected leader, sending a full tablet report...
I20260808 11:38:22.717511 4337 ts_tablet_manager.cc:1428] T e760ac585f944970b77d88b082f507da P c4d17c1c8b43495f81c16293bc8986f0: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20260808 11:38:22.720805 4346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.719833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.24.76.67:50717)
0808 11:38:22.720397 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:22.720563 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"threads_started":1}
I20260808 11:38:22.721143 4343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.710176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.24.76.66:51705)
0808 11:38:22.720761 (+ 10585us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:22.720925 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10454,"threads_started":1}
I20260808 11:38:22.722141 4346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.721750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.24.76.67:53959)
0808 11:38:22.721920 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:22.722013 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 11:38:22.722257 4343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.721851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.24.76.66:38459)
0808 11:38:22.722068 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:22.722140 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
I20260808 11:38:22.722460 4345 tablet_bootstrap.cc:492] T e760ac585f944970b77d88b082f507da P 900623cd4f4a40c99098e6dfbe9f684f: Bootstrap starting.
I20260808 11:38:22.723109 4196 heartbeater.cc:503] Master 127.24.76.125:42897 was elected leader, sending a full tablet report...
I20260808 11:38:22.724071 4345 tablet_bootstrap.cc:654] T e760ac585f944970b77d88b082f507da P 900623cd4f4a40c99098e6dfbe9f684f: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:22.725986 4345 tablet_bootstrap.cc:492] T e760ac585f944970b77d88b082f507da P 900623cd4f4a40c99098e6dfbe9f684f: No bootstrap required, opened a new log
I20260808 11:38:22.726104 4345 ts_tablet_manager.cc:1397] T e760ac585f944970b77d88b082f507da P 900623cd4f4a40c99098e6dfbe9f684f: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260808 11:38:22.726722 4345 raft_consensus.cc:348] T e760ac585f944970b77d88b082f507da P 900623cd4f4a40c99098e6dfbe9f684f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d17c1c8b43495f81c16293bc8986f0" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 43707 } } peers { permanent_uuid: "9b038a3b88794c9e8a63d670257a770d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44933 } } peers { permanent_uuid: "900623cd4f4a40c99098e6dfbe9f684f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 33613 } }
I20260808 11:38:22.726886 4345 raft_consensus.cc:374] T e760ac585f944970b77d88b082f507da P 900623cd4f4a40c99098e6dfbe9f684f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:22.726949 4345 raft_consensus.cc:729] T e760ac585f944970b77d88b082f507da P 900623cd4f4a40c99098e6dfbe9f684f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 900623cd4f4a40c99098e6dfbe9f684f, State: Initialized, Role: FOLLOWER
I20260808 11:38:22.727335 4345 consensus_queue.cc:260] T e760ac585f944970b77d88b082f507da P 900623cd4f4a40c99098e6dfbe9f684f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d17c1c8b43495f81c16293bc8986f0" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 43707 } } peers { permanent_uuid: "9b038a3b88794c9e8a63d670257a770d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44933 } } peers { permanent_uuid: "900623cd4f4a40c99098e6dfbe9f684f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 33613 } }
I20260808 11:38:22.727582 4348 raft_consensus.cc:482] T e760ac585f944970b77d88b082f507da P 900623cd4f4a40c99098e6dfbe9f684f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:38:22.727778 4348 raft_consensus.cc:504] T e760ac585f944970b77d88b082f507da P 900623cd4f4a40c99098e6dfbe9f684f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d17c1c8b43495f81c16293bc8986f0" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 43707 } } peers { permanent_uuid: "9b038a3b88794c9e8a63d670257a770d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44933 } } peers { permanent_uuid: "900623cd4f4a40c99098e6dfbe9f684f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 33613 } }
I20260808 11:38:22.728152 4345 ts_tablet_manager.cc:1428] T e760ac585f944970b77d88b082f507da P 900623cd4f4a40c99098e6dfbe9f684f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 11:38:22.734918 4350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.728454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.24.76.65:53889)
0808 11:38:22.734407 (+ 5953us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:22.734604 (+ 197us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5823,"thread_start_us":5696,"threads_started":1}
I20260808 11:38:22.736387 4348 leader_election.cc:288] T e760ac585f944970b77d88b082f507da P 900623cd4f4a40c99098e6dfbe9f684f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d17c1c8b43495f81c16293bc8986f0 (127.24.76.67:43707), 9b038a3b88794c9e8a63d670257a770d (127.24.76.66:44933)
I20260808 11:38:22.742107 3752 heartbeater.cc:503] Master 127.24.76.125:42897 was elected leader, sending a full tablet report...
I20260808 11:38:22.742549 4355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.735586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.24.76.65:51729)
0808 11:38:22.742194 (+ 6608us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:22.742349 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6482}
I20260808 11:38:22.743194 4331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.730553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:41537 (local address 127.24.76.67:43707)
0808 11:38:22.730739 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:22.730746 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:22.735299 (+ 4553us) server_negotiation.cc:407] Connection header received
0808 11:38:22.735573 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:22.735580 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:22.735665 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:22.735725 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:38:22.736461 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:22.737590 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:22.740195 (+ 2605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:22.741394 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:22.741542 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:22.741621 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:22.742842 (+ 1221us) server_negotiation.cc:299] Negotiation successful
0808 11:38:22.742977 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 11:38:22.744263 4350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.730148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:43707 (local address 127.24.76.65:41537)
0808 11:38:22.735186 (+ 5038us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:22.735212 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:22.735407 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:22.735922 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:22.735928 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:22.735964 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:38:22.736288 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:22.736300 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:22.737728 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:22.737734 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:22.740031 (+ 2297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:22.740049 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:22.741591 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:22.741598 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:22.741990 (+ 392us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:22.742026 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 11:38:22.743557 (+ 1531us) client_negotiation.cc:240] Negotiation successful
0808 11:38:22.743709 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4946,"thread_start_us":11600,"threads_started":1}
I20260808 11:38:22.745046 4121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e760ac585f944970b77d88b082f507da" candidate_uuid: "900623cd4f4a40c99098e6dfbe9f684f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d17c1c8b43495f81c16293bc8986f0" is_pre_election: true
I20260808 11:38:22.745389 4121 raft_consensus.cc:2454] T e760ac585f944970b77d88b082f507da P c4d17c1c8b43495f81c16293bc8986f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 900623cd4f4a40c99098e6dfbe9f684f in term 0.
I20260808 11:38:22.745918 3586 leader_election.cc:302] T e760ac585f944970b77d88b082f507da P 900623cd4f4a40c99098e6dfbe9f684f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 900623cd4f4a40c99098e6dfbe9f684f, c4d17c1c8b43495f81c16293bc8986f0; no voters:
I20260808 11:38:22.746315 4348 raft_consensus.cc:2781] T e760ac585f944970b77d88b082f507da P 900623cd4f4a40c99098e6dfbe9f684f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:38:22.746470 4348 raft_consensus.cc:482] T e760ac585f944970b77d88b082f507da P 900623cd4f4a40c99098e6dfbe9f684f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:38:22.746559 4348 raft_consensus.cc:3037] T e760ac585f944970b77d88b082f507da P 900623cd4f4a40c99098e6dfbe9f684f [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:22.748678 4332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.730541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:40321 (local address 127.24.76.66:44933)
0808 11:38:22.730743 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:22.730748 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:22.741496 (+ 10748us) server_negotiation.cc:407] Connection header received
0808 11:38:22.742895 (+ 1399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:22.742901 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:22.742971 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:22.743022 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:38:22.743894 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:22.744942 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:22.746940 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:22.747802 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:22.747917 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:22.747973 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:22.748357 (+ 384us) server_negotiation.cc:299] Negotiation successful
0808 11:38:22.748443 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 11:38:22.748867 4352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.730185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:44933 (local address 127.24.76.65:40321)
0808 11:38:22.741398 (+ 11213us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:22.741415 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:22.741805 (+ 390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:22.743205 (+ 1400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:22.743210 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:22.743233 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:38:22.743592 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:22.743606 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:22.745061 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:22.745066 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:22.746787 (+ 1721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:22.746802 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:22.748022 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:22.748029 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:22.748217 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:22.748250 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 11:38:22.748342 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 11:38:22.748457 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11094,"thread_start_us":11461,"threads_started":1}
I20260808 11:38:22.749230 4348 raft_consensus.cc:504] T e760ac585f944970b77d88b082f507da P 900623cd4f4a40c99098e6dfbe9f684f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d17c1c8b43495f81c16293bc8986f0" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 43707 } } peers { permanent_uuid: "9b038a3b88794c9e8a63d670257a770d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44933 } } peers { permanent_uuid: "900623cd4f4a40c99098e6dfbe9f684f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 33613 } }
I20260808 11:38:22.749472 3904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e760ac585f944970b77d88b082f507da" candidate_uuid: "900623cd4f4a40c99098e6dfbe9f684f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b038a3b88794c9e8a63d670257a770d" is_pre_election: true
I20260808 11:38:22.749686 3904 raft_consensus.cc:2454] T e760ac585f944970b77d88b082f507da P 9b038a3b88794c9e8a63d670257a770d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 900623cd4f4a40c99098e6dfbe9f684f in term 0.
I20260808 11:38:22.749835 4348 leader_election.cc:288] T e760ac585f944970b77d88b082f507da P 900623cd4f4a40c99098e6dfbe9f684f [CANDIDATE]: Term 1 election: Requested vote from peers c4d17c1c8b43495f81c16293bc8986f0 (127.24.76.67:43707), 9b038a3b88794c9e8a63d670257a770d (127.24.76.66:44933)
I20260808 11:38:22.750094 4121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e760ac585f944970b77d88b082f507da" candidate_uuid: "900623cd4f4a40c99098e6dfbe9f684f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d17c1c8b43495f81c16293bc8986f0"
I20260808 11:38:22.750288 4121 raft_consensus.cc:3037] T e760ac585f944970b77d88b082f507da P c4d17c1c8b43495f81c16293bc8986f0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:22.750495 3904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e760ac585f944970b77d88b082f507da" candidate_uuid: "900623cd4f4a40c99098e6dfbe9f684f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b038a3b88794c9e8a63d670257a770d"
I20260808 11:38:22.750695 3904 raft_consensus.cc:3037] T e760ac585f944970b77d88b082f507da P 9b038a3b88794c9e8a63d670257a770d [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:22.753244 4121 raft_consensus.cc:2454] T e760ac585f944970b77d88b082f507da P c4d17c1c8b43495f81c16293bc8986f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 900623cd4f4a40c99098e6dfbe9f684f in term 1.
I20260808 11:38:22.753520 3904 raft_consensus.cc:2454] T e760ac585f944970b77d88b082f507da P 9b038a3b88794c9e8a63d670257a770d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 900623cd4f4a40c99098e6dfbe9f684f in term 1.
I20260808 11:38:22.753615 3586 leader_election.cc:302] T e760ac585f944970b77d88b082f507da P 900623cd4f4a40c99098e6dfbe9f684f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 900623cd4f4a40c99098e6dfbe9f684f, c4d17c1c8b43495f81c16293bc8986f0; no voters:
I20260808 11:38:22.753908 4348 raft_consensus.cc:2781] T e760ac585f944970b77d88b082f507da P 900623cd4f4a40c99098e6dfbe9f684f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:38:22.754069 4348 raft_consensus.cc:686] T e760ac585f944970b77d88b082f507da P 900623cd4f4a40c99098e6dfbe9f684f [term 1 LEADER]: Becoming Leader. State: Replica: 900623cd4f4a40c99098e6dfbe9f684f, State: Running, Role: LEADER
I20260808 11:38:22.754437 4348 consensus_queue.cc:237] T e760ac585f944970b77d88b082f507da P 900623cd4f4a40c99098e6dfbe9f684f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d17c1c8b43495f81c16293bc8986f0" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 43707 } } peers { permanent_uuid: "9b038a3b88794c9e8a63d670257a770d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44933 } } peers { permanent_uuid: "900623cd4f4a40c99098e6dfbe9f684f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 33613 } }
I20260808 11:38:22.754906 4352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.754579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.24.76.65:34929)
0808 11:38:22.754710 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:22.754797 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 11:38:22.758023 3181 catalog_manager.cc:5495] T e760ac585f944970b77d88b082f507da P 900623cd4f4a40c99098e6dfbe9f684f reported cstate change: term changed from 0 to 1, leader changed from <none> to 900623cd4f4a40c99098e6dfbe9f684f (127.24.76.65). New cstate: current_term: 1 leader_uuid: "900623cd4f4a40c99098e6dfbe9f684f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4d17c1c8b43495f81c16293bc8986f0" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 43707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b038a3b88794c9e8a63d670257a770d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "900623cd4f4a40c99098e6dfbe9f684f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 33613 } health_report { overall_health: HEALTHY } } }
I20260808 11:38:22.789927 4335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.784840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59224 (local address 127.24.76.65:33613)
0808 11:38:22.785020 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:22.785027 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:22.785104 (+ 77us) server_negotiation.cc:407] Connection header received
0808 11:38:22.785330 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:22.785334 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:22.785415 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:22.785552 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:38:22.785938 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:22.786626 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:22.787891 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:22.788307 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:22.788415 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:22.788834 (+ 419us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:38:22.789342 (+ 508us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:38:22.789469 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:22.789586 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 11:38:22.789662 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 11:38:22.793726 3904 raft_consensus.cc:1261] T e760ac585f944970b77d88b082f507da P 9b038a3b88794c9e8a63d670257a770d [term 1 FOLLOWER]: Refusing update from remote peer 900623cd4f4a40c99098e6dfbe9f684f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:38:22.794100 4121 raft_consensus.cc:1261] T e760ac585f944970b77d88b082f507da P c4d17c1c8b43495f81c16293bc8986f0 [term 1 FOLLOWER]: Refusing update from remote peer 900623cd4f4a40c99098e6dfbe9f684f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:38:22.794528 4349 consensus_queue.cc:1035] T e760ac585f944970b77d88b082f507da P 900623cd4f4a40c99098e6dfbe9f684f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b038a3b88794c9e8a63d670257a770d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 44933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:38:22.794898 4348 consensus_queue.cc:1035] T e760ac585f944970b77d88b082f507da P 900623cd4f4a40c99098e6dfbe9f684f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4d17c1c8b43495f81c16293bc8986f0" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 43707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:38:22.797772 4343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.796937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.24.76.66:60445)
0808 11:38:22.797386 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:22.797513 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316}
I20260808 11:38:22.799751 4346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.799298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.24.76.67:50369)
0808 11:38:22.799475 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:22.799581 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 11:38:22.802773 4343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.802300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.24.76.66:45911)
0808 11:38:22.802522 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:22.802608 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260808 11:38:22.809960 4352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.809344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.24.76.65:43083)
0808 11:38:22.809656 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:22.809759 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191}
I20260808 11:38:22.814503 4352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.812820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.24.76.65:35187)
0808 11:38:22.814223 (+ 1403us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:22.814321 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1286}
I20260808 11:38:22.816516 4346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.813980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.24.76.67:43145)
0808 11:38:22.816255 (+ 2275us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:22.816356 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2167}
I20260808 11:38:22.821408 3179 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:38:22.821664 3179 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:38:22.827272 3179 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 79a1352c611a496ba0577480a04e026f: Sending DeleteTablet for 3 replicas of tablet e760ac585f944970b77d88b082f507da
I20260808 11:38:22.828243 4093 tablet_service.cc:1514] Processing DeleteTablet for tablet e760ac585f944970b77d88b082f507da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:38:22 UTC) from {username='oryx', principal='oryx/127.24.76.125@KRBTEST.COM'} at 127.0.0.1:59488
I20260808 11:38:22.828405 3648 tablet_service.cc:1514] Processing DeleteTablet for tablet e760ac585f944970b77d88b082f507da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:38:22 UTC) from {username='oryx', principal='oryx/127.24.76.125@KRBTEST.COM'} at 127.0.0.1:59210
I20260808 11:38:22.828645 3873 tablet_service.cc:1514] Processing DeleteTablet for tablet e760ac585f944970b77d88b082f507da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:38:22 UTC) from {username='oryx', principal='oryx/127.24.76.125@KRBTEST.COM'} at 127.0.0.1:54548
I20260808 11:38:22.830708 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 3445
I20260808 11:38:22.844331 4375 tablet_replica.cc:330] T e760ac585f944970b77d88b082f507da P c4d17c1c8b43495f81c16293bc8986f0: stopping tablet replica
I20260808 11:38:22.844535 4375 raft_consensus.cc:2229] T e760ac585f944970b77d88b082f507da P c4d17c1c8b43495f81c16293bc8986f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:38:22.844839 4375 raft_consensus.cc:2258] T e760ac585f944970b77d88b082f507da P c4d17c1c8b43495f81c16293bc8986f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:38:22.848016 4375 ts_tablet_manager.cc:1905] T e760ac585f944970b77d88b082f507da P c4d17c1c8b43495f81c16293bc8986f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:38:22.858357 4378 tablet_replica.cc:330] T e760ac585f944970b77d88b082f507da P 9b038a3b88794c9e8a63d670257a770d: stopping tablet replica
I20260808 11:38:22.858434 4375 ts_tablet_manager.cc:1918] T e760ac585f944970b77d88b082f507da P c4d17c1c8b43495f81c16293bc8986f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:38:22.858589 4375 log.cc:1196] T e760ac585f944970b77d88b082f507da P c4d17c1c8b43495f81c16293bc8986f0: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-2/wal/wals/e760ac585f944970b77d88b082f507da
I20260808 11:38:22.858790 4378 raft_consensus.cc:2229] T e760ac585f944970b77d88b082f507da P 9b038a3b88794c9e8a63d670257a770d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:38:22.859110 4375 ts_tablet_manager.cc:1939] T e760ac585f944970b77d88b082f507da P c4d17c1c8b43495f81c16293bc8986f0: Deleting consensus metadata
I20260808 11:38:22.859661 4378 raft_consensus.cc:2258] T e760ac585f944970b77d88b082f507da P 9b038a3b88794c9e8a63d670257a770d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:38:22.860483 4376 tablet_replica.cc:330] T e760ac585f944970b77d88b082f507da P 900623cd4f4a40c99098e6dfbe9f684f: stopping tablet replica
I20260808 11:38:22.860577 3164 catalog_manager.cc:4841] TS c4d17c1c8b43495f81c16293bc8986f0 (127.24.76.67:43707): tablet e760ac585f944970b77d88b082f507da (table test-table [id=2727f60fe07e42c5a23bda57e184d76f]) successfully deleted
I20260808 11:38:22.860747 4376 raft_consensus.cc:2229] T e760ac585f944970b77d88b082f507da P 900623cd4f4a40c99098e6dfbe9f684f [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:38:22.861768 4378 ts_tablet_manager.cc:1905] T e760ac585f944970b77d88b082f507da P 9b038a3b88794c9e8a63d670257a770d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:38:22.863206 4376 raft_consensus.cc:2258] T e760ac585f944970b77d88b082f507da P 900623cd4f4a40c99098e6dfbe9f684f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:38:22.867051 4378 ts_tablet_manager.cc:1918] T e760ac585f944970b77d88b082f507da P 9b038a3b88794c9e8a63d670257a770d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:38:22.867183 4378 log.cc:1196] T e760ac585f944970b77d88b082f507da P 9b038a3b88794c9e8a63d670257a770d: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-1/wal/wals/e760ac585f944970b77d88b082f507da
I20260808 11:38:22.867604 4378 ts_tablet_manager.cc:1939] T e760ac585f944970b77d88b082f507da P 9b038a3b88794c9e8a63d670257a770d: Deleting consensus metadata
I20260808 11:38:22.868777 3163 catalog_manager.cc:4841] TS 9b038a3b88794c9e8a63d670257a770d (127.24.76.66:44933): tablet e760ac585f944970b77d88b082f507da (table test-table [id=2727f60fe07e42c5a23bda57e184d76f]) successfully deleted
I20260808 11:38:22.869324 4376 ts_tablet_manager.cc:1905] T e760ac585f944970b77d88b082f507da P 900623cd4f4a40c99098e6dfbe9f684f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:38:22.874398 4335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:22.831737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59230 (local address 127.24.76.65:33613)
0808 11:38:22.831918 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:22.831925 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:22.856084 (+ 24159us) server_negotiation.cc:407] Connection header received
0808 11:38:22.856286 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:22.856292 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:22.856380 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:22.856504 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:22.859612 (+ 3108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:22.860794 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:22.862657 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:22.863361 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:22.863507 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:22.866895 (+ 3388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:22.866911 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:22.866932 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:22.866975 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:22.871131 (+ 4156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:22.872004 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:22.872015 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:22.872019 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:22.872237 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:22.872654 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:22.872659 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:22.872663 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:22.873116 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:22.873467 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:22.873746 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 11:38:22.873997 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 11:38:22.875103 3740 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:22.984061 4376 ts_tablet_manager.cc:1918] T e760ac585f944970b77d88b082f507da P 900623cd4f4a40c99098e6dfbe9f684f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:38:22.984318 4376 log.cc:1196] T e760ac585f944970b77d88b082f507da P 900623cd4f4a40c99098e6dfbe9f684f: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786189042529715-24881-0/minicluster-data/ts-0/wal/wals/e760ac585f944970b77d88b082f507da
I20260808 11:38:22.984684 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 3445
I20260808 11:38:23.006165 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 3768
I20260808 11:38:23.006924 4333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:23.006523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:33613 (local address 127.0.0.1:59246)
0808 11:38:23.006680 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:23.006758 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:33613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 11:38:23.007272 3164 catalog_manager.cc:4584] TS 900623cd4f4a40c99098e6dfbe9f684f (127.24.76.65:33613): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e760ac585f944970b77d88b082f507da: Network error: Client connection negotiation failed: client connection to 127.24.76.65:33613: connect: Connection refused (error 111)
I20260808 11:38:23.018955 4332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:23.007059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54562 (local address 127.24.76.66:44933)
0808 11:38:23.007250 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:23.007257 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:23.007279 (+ 22us) server_negotiation.cc:407] Connection header received
0808 11:38:23.007493 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:23.007499 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:23.007617 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:23.007700 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:23.009042 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:23.009850 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:23.011112 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:23.011493 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:23.011581 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:23.013854 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:23.013869 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:23.013886 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:23.013950 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:23.016589 (+ 2639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:23.017139 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:23.017147 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:23.017149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:23.017262 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:23.017533 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:23.017536 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:23.017538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:23.017865 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:23.018288 (+ 423us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:23.018474 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 11:38:23.018638 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 11:38:23.019557 3971 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:23.067677 4333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:23.067231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:33613 (local address 127.0.0.1:59260)
0808 11:38:23.067418 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:23.067518 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:33613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 11:38:23.068683 4333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:23.068353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:33613 (local address 127.0.0.1:59266)
0808 11:38:23.068488 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:23.068572 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:33613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 11:38:23.111223 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 3768
I20260808 11:38:23.123632 4333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:23.123195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.0.0.1:51458)
0808 11:38:23.123379 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:23.123486 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 11:38:23.124724 4333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:23.124365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44823 (local address 127.0.0.1:51474)
0808 11:38:23.124533 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:23.124615 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 11:38:23.134910 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 3994
I20260808 11:38:23.137053 4333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:23.136651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:33613 (local address 127.0.0.1:59280)
0808 11:38:23.136819 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:23.136912 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:33613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 11:38:23.138082 4333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:23.137694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:33613 (local address 127.0.0.1:59284)
0808 11:38:23.137842 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:23.137947 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:33613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 11:38:23.147743 4331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:23.135646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59496 (local address 127.24.76.67:43707)
0808 11:38:23.135815 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:23.135820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:23.135839 (+ 19us) server_negotiation.cc:407] Connection header received
0808 11:38:23.136055 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:23.136060 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:23.136148 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:23.136262 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:23.137604 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:23.138565 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:23.139742 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:23.140125 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:23.140213 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:23.142609 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:23.142624 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:23.142642 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:23.142676 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:23.145487 (+ 2811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:23.146042 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:23.146048 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:23.146050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:23.146182 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:23.146443 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:23.146445 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:23.146447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:23.146790 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:23.147029 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:23.147276 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 11:38:23.147442 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 11:38:23.148344 4182 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:23.239578 4333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:23.239097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:33613 (local address 127.0.0.1:59290)
0808 11:38:23.239322 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:23.239417 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:33613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260808 11:38:23.240720 4333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:23.240341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:33613 (local address 127.0.0.1:59304)
0808 11:38:23.240504 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:23.240603 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:33613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 11:38:23.266695 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 3994
I20260808 11:38:23.296568 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 2983
I20260808 11:38:23.297778 3224 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:23.423970 4333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:23.423326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:33613 (local address 127.0.0.1:59314)
0808 11:38:23.423581 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:23.423699 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:33613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260808 11:38:23.424319 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 2983
W20260808 11:38:23.450213 3441 connection.cc:535] server connection from 127.0.0.1:39498 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 11:38:23.450593 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 3127
I20260808 11:38:23.451642 3524 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:23.567399 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 3127
2026-08-08T11:38:23Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9924 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:38:23 dist-test-slave-tc2s krb5kdc[4388](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:38:23 dist-test-slave-tc2s krb5kdc[4388](info): set up 2 sockets
Aug 08 11:38:23 dist-test-slave-tc2s krb5kdc[4388](info): commencing operation
krb5kdc: starting...
W20260808 11:38:24.770696 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.003s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:38:24 dist-test-slave-tc2s krb5kdc[4388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189104, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:38:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:38:24Z Disabled control of system clock
WARNING: no policy specified for oryx/127.24.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.126.keytab.
Entry for principal oryx/127.24.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.126.keytab.
I20260808 11:38:25.009078 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:33541
--webserver_interface=127.24.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.126.keytab
--principal=oryx/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:38147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:33541
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:38:25.330245 4409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:25.330528 4409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:25.330660 4409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:25.337538 4409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:38:25.337638 4409 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 11:38:25.337706 4409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:25.337787 4409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:25.337854 4409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:38:25.337955 4409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:38:25.347640 4409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:38147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:33541
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.126.keytab
--principal=oryx/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:33541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:25.348860 4409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:25.350169 4409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:25.361735 4417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:25.361646 4418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:25.361673 4420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:25.362510 4409 server_base.cc:1034] running on GCE node
I20260808 11:38:25.363243 4409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:25.364526 4409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:25.365754 4409 hybrid_clock.cc:648] HybridClock initialized: now 1786189105365679 us; error 47 us; skew 500 ppm
I20260808 11:38:25.366091 4409 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:38:25 dist-test-slave-tc2s krb5kdc[4388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189105, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:25.369139 4409 init.cc:377] Logged in from keytab as oryx/127.24.76.126@KRBTEST.COM (short username oryx)
I20260808 11:38:25.370500 4409 webserver.cc:460] Webserver started at http://127.24.76.126:38043/ using document root <none> and password file <none>
I20260808 11:38:25.371165 4409 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:25.371264 4409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:25.371567 4409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:25.374421 4409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "508408a84d5c4b838a2198a913aff5a2"
format_stamp: "Formatted at 2026-08-08 11:38:25 on dist-test-slave-tc2s"
server_key: "f7420d8ecc93fca675900ca6a483daee"
server_key_iv: "6e885330b9709982f871b49528672e84"
server_key_version: "encryptionkey@0"
I20260808 11:38:25.375183 4409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "508408a84d5c4b838a2198a913aff5a2"
format_stamp: "Formatted at 2026-08-08 11:38:25 on dist-test-slave-tc2s"
server_key: "f7420d8ecc93fca675900ca6a483daee"
server_key_iv: "6e885330b9709982f871b49528672e84"
server_key_version: "encryptionkey@0"
I20260808 11:38:25.379909 4409 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 11:38:25.383126 4436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:25.384284 4409 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 11:38:25.384438 4409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "508408a84d5c4b838a2198a913aff5a2"
format_stamp: "Formatted at 2026-08-08 11:38:25 on dist-test-slave-tc2s"
server_key: "f7420d8ecc93fca675900ca6a483daee"
server_key_iv: "6e885330b9709982f871b49528672e84"
server_key_version: "encryptionkey@0"
I20260808 11:38:25.384629 4409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:25.412202 4409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:25.417783 4409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:25.418048 4409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:25.437939 4409 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:33541
I20260808 11:38:25.437954 4521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:33541 every 8 connection(s)
I20260808 11:38:25.439272 4409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:38:25.442901 4523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:25.443387 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 4409
I20260808 11:38:25.443578 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:38:25.444214 24881 external_mini_cluster.cc:1428] Setting key dd6827a4e6b9d68c5fba268c8ea9f0c4
I20260808 11:38:25.456517 4523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 508408a84d5c4b838a2198a913aff5a2: Bootstrap starting.
I20260808 11:38:25.459378 4523 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 508408a84d5c4b838a2198a913aff5a2: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:25.460364 4523 log.cc:824] T 00000000000000000000000000000000 P 508408a84d5c4b838a2198a913aff5a2: Log is configured to *not* fsync() on all Append() calls
Aug 08 11:38:25 dist-test-slave-tc2s krb5kdc[4388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189104, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM
I20260808 11:38:25.463025 4523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 508408a84d5c4b838a2198a913aff5a2: No bootstrap required, opened a new log
I20260808 11:38:25.471937 4527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:25.446153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46320 (local address 127.24.76.126:33541)
0808 11:38:25.446929 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:25.446944 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:25.447021 (+ 77us) server_negotiation.cc:407] Connection header received
0808 11:38:25.449105 (+ 2084us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:25.449133 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:25.449605 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:25.450094 (+ 489us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:25.451324 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:25.453273 (+ 1949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:25.454541 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:25.455553 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:25.455720 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:25.463760 (+ 8040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:25.463771 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:25.463789 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:25.463832 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:25.467629 (+ 3797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:25.468257 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:25.468265 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:25.468277 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:25.468491 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:25.468808 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:25.468813 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:25.468816 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:25.469500 (+ 684us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:25.469802 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:25.470133 (+ 331us) server_negotiation.cc:299] Negotiation successful
0808 11:38:25.470563 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":302,"threads_started":1}
I20260808 11:38:25.473047 4523 raft_consensus.cc:348] T 00000000000000000000000000000000 P 508408a84d5c4b838a2198a913aff5a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "508408a84d5c4b838a2198a913aff5a2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 33541 } }
I20260808 11:38:25.473374 4523 raft_consensus.cc:374] T 00000000000000000000000000000000 P 508408a84d5c4b838a2198a913aff5a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:25.473486 4523 raft_consensus.cc:729] T 00000000000000000000000000000000 P 508408a84d5c4b838a2198a913aff5a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 508408a84d5c4b838a2198a913aff5a2, State: Initialized, Role: FOLLOWER
I20260808 11:38:25.474097 4523 consensus_queue.cc:260] T 00000000000000000000000000000000 P 508408a84d5c4b838a2198a913aff5a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "508408a84d5c4b838a2198a913aff5a2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 33541 } }
I20260808 11:38:25.474324 4523 raft_consensus.cc:388] T 00000000000000000000000000000000 P 508408a84d5c4b838a2198a913aff5a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:38:25.474464 4523 raft_consensus.cc:482] T 00000000000000000000000000000000 P 508408a84d5c4b838a2198a913aff5a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:38:25.474622 4523 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 508408a84d5c4b838a2198a913aff5a2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:25.476688 4523 raft_consensus.cc:504] T 00000000000000000000000000000000 P 508408a84d5c4b838a2198a913aff5a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "508408a84d5c4b838a2198a913aff5a2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 33541 } }
I20260808 11:38:25.477160 4523 leader_election.cc:302] T 00000000000000000000000000000000 P 508408a84d5c4b838a2198a913aff5a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 508408a84d5c4b838a2198a913aff5a2; no voters:
I20260808 11:38:25.477676 4523 leader_election.cc:288] T 00000000000000000000000000000000 P 508408a84d5c4b838a2198a913aff5a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:38:25.477794 4533 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 508408a84d5c4b838a2198a913aff5a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:38:25.479444 4533 raft_consensus.cc:686] T 00000000000000000000000000000000 P 508408a84d5c4b838a2198a913aff5a2 [term 1 LEADER]: Becoming Leader. State: Replica: 508408a84d5c4b838a2198a913aff5a2, State: Running, Role: LEADER
I20260808 11:38:25.480032 4523 sys_catalog.cc:564] T 00000000000000000000000000000000 P 508408a84d5c4b838a2198a913aff5a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:38:25.479912 4533 consensus_queue.cc:237] T 00000000000000000000000000000000 P 508408a84d5c4b838a2198a913aff5a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "508408a84d5c4b838a2198a913aff5a2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 33541 } }
I20260808 11:38:25.482873 4538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 508408a84d5c4b838a2198a913aff5a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 508408a84d5c4b838a2198a913aff5a2. Latest consensus state: current_term: 1 leader_uuid: "508408a84d5c4b838a2198a913aff5a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "508408a84d5c4b838a2198a913aff5a2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 33541 } } }
I20260808 11:38:25.483233 4538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 508408a84d5c4b838a2198a913aff5a2 [sys.catalog]: This master's current role is: LEADER
I20260808 11:38:25.483459 4536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 508408a84d5c4b838a2198a913aff5a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "508408a84d5c4b838a2198a913aff5a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "508408a84d5c4b838a2198a913aff5a2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 33541 } } }
I20260808 11:38:25.483727 4536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 508408a84d5c4b838a2198a913aff5a2 [sys.catalog]: This master's current role is: LEADER
I20260808 11:38:25.485085 4553 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:38:25.490828 4553 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:38:25.500192 4553 catalog_manager.cc:1347] Generated new cluster ID: d48950011eac4d56b37154ada0923efa
I20260808 11:38:25.500300 4553 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:38:25.548435 4553 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:38:25.549743 4553 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:38:25.556761 4553 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 508408a84d5c4b838a2198a913aff5a2: Generated new TSK 0
I20260808 11:38:25.557483 4553 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.24.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.65.keytab.
Entry for principal oryx/127.24.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.65.keytab.
I20260808 11:38:25.618770 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.65:0
--local_ip_for_outbound_sockets=127.24.76.65
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:33541
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.65.keytab
--principal=oryx/127.24.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:38147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:38:25.957177 4575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:25.957481 4575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:25.957623 4575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:25.964605 4575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:25.964699 4575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:25.964818 4575 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 11:38:25.964918 4575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.65
I20260808 11:38:25.974339 4575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:38147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.65.keytab
--principal=oryx/127.24.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.76.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.76.126:33541
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.65
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:25.975498 4575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:25.976763 4575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:25.989530 4583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:25.989562 4584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:25.989715 4586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:25.990536 4575 server_base.cc:1034] running on GCE node
I20260808 11:38:25.991074 4575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:25.991928 4575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:25.993147 4575 hybrid_clock.cc:648] HybridClock initialized: now 1786189105993084 us; error 29 us; skew 500 ppm
I20260808 11:38:25.993393 4575 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:38:25 dist-test-slave-tc2s krb5kdc[4388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189105, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:25.996333 4575 init.cc:377] Logged in from keytab as oryx/127.24.76.65@KRBTEST.COM (short username oryx)
I20260808 11:38:25.997650 4575 webserver.cc:460] Webserver started at http://127.24.76.65:44065/ using document root <none> and password file <none>
I20260808 11:38:25.998323 4575 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:25.998414 4575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:25.998842 4575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:26.001658 4575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-0/data/instance:
uuid: "5a6215971eda4de2bbb2fbf88c75d70f"
format_stamp: "Formatted at 2026-08-08 11:38:25 on dist-test-slave-tc2s"
server_key: "915928c07df656abb37bb1f71de77d53"
server_key_iv: "2c3df4761d739bc002e57b64705737e0"
server_key_version: "encryptionkey@0"
I20260808 11:38:26.002449 4575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance:
uuid: "5a6215971eda4de2bbb2fbf88c75d70f"
format_stamp: "Formatted at 2026-08-08 11:38:25 on dist-test-slave-tc2s"
server_key: "915928c07df656abb37bb1f71de77d53"
server_key_iv: "2c3df4761d739bc002e57b64705737e0"
server_key_version: "encryptionkey@0"
I20260808 11:38:26.007205 4575 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260808 11:38:26.010473 4598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:26.011664 4575 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:38:26.011806 4575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-0/wal
uuid: "5a6215971eda4de2bbb2fbf88c75d70f"
format_stamp: "Formatted at 2026-08-08 11:38:25 on dist-test-slave-tc2s"
server_key: "915928c07df656abb37bb1f71de77d53"
server_key_iv: "2c3df4761d739bc002e57b64705737e0"
server_key_version: "encryptionkey@0"
I20260808 11:38:26.012032 4575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:26.058073 4575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:26.062480 4575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:26.062740 4575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:26.065367 4575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:38:26.065451 4575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:26.065590 4575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:38:26.065676 4575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 11:38:26 dist-test-slave-tc2s krb5kdc[4388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189105, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM
I20260808 11:38:26.087697 4610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:26.065988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:57391 (local address 127.24.76.126:33541)
0808 11:38:26.066599 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:26.066605 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:26.067628 (+ 1023us) server_negotiation.cc:407] Connection header received
0808 11:38:26.069994 (+ 2366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:26.069999 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:26.070067 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:26.070142 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:26.072434 (+ 2292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:26.073578 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:26.075781 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:26.076315 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:26.076434 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:26.081169 (+ 4735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:26.081176 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:26.081180 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:26.081215 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:26.084231 (+ 3016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:26.085070 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:26.085075 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:26.085080 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:26.085228 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:26.085871 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:26.085877 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:26.085881 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:26.086257 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:26.086394 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:26.087112 (+ 718us) server_negotiation.cc:299] Negotiation successful
0808 11:38:26.087330 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":365,"threads_started":1}
I20260808 11:38:26.088892 4611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:26.066631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33541 (local address 127.24.76.65:57391)
0808 11:38:26.067462 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:26.067509 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:26.069603 (+ 2094us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:26.070622 (+ 1019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:26.070640 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:26.071395 (+ 755us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:26.072206 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:26.072235 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:26.073751 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:26.073760 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:26.075587 (+ 1827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:26.075608 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:26.076571 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:26.076579 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:26.076830 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:26.077578 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:26.077602 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:26.080962 (+ 3360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:26.084360 (+ 3398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:26.084368 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:26.084384 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:26.084949 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:26.085360 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:26.085365 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:26.085369 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:26.085724 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:26.086391 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:26.086400 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:26.086749 (+ 349us) client_negotiation.cc:769] Sending connection context
0808 11:38:26.086975 (+ 226us) client_negotiation.cc:240] Negotiation successful
0808 11:38:26.087507 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":350,"threads_started":1}
I20260808 11:38:26.109822 4725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:26.097009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33541 (local address 127.24.76.65:54127)
0808 11:38:26.097570 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:26.097585 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:26.097709 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:26.098054 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:26.098058 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:26.098695 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:26.099243 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:26.099258 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:26.100482 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:26.100488 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:26.101501 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:26.101513 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:26.102295 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:26.102302 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:26.102497 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:26.103131 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:26.103147 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:26.103878 (+ 731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:26.107139 (+ 3261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:26.107144 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:26.107148 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:26.107658 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:26.108130 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:26.108135 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:26.108139 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:26.108309 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:26.108979 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:26.108983 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:26.109137 (+ 154us) client_negotiation.cc:769] Sending connection context
0808 11:38:26.109259 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 11:38:26.109502 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":301,"threads_started":1}
I20260808 11:38:26.110013 4610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:26.096928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:54127 (local address 127.24.76.126:33541)
0808 11:38:26.097076 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:26.097080 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:26.097668 (+ 588us) server_negotiation.cc:407] Connection header received
0808 11:38:26.097819 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:26.097824 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:26.097923 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:26.098010 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:26.099387 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:26.100355 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:26.101633 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:26.102102 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:26.102209 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:26.104061 (+ 1852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:26.104067 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:26.104072 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:26.104111 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:26.107019 (+ 2908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:26.107823 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:26.107828 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:26.107831 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:26.107997 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:26.108452 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:26.108458 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:26.108462 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:26.108854 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:26.109005 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:26.109380 (+ 375us) server_negotiation.cc:299] Negotiation successful
0808 11:38:26.109633 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 11:38:26.116556 4575 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.65:43047
I20260808 11:38:26.116607 4787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.65:43047 every 8 connection(s)
I20260808 11:38:26.117686 4575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
I20260808 11:38:26.119593 4789 heartbeater.cc:347] Connected to a master server at 127.24.76.126:33541
I20260808 11:38:26.119819 4789 heartbeater.cc:464] Registering TS with master...
I20260808 11:38:26.120419 4789 heartbeater.cc:511] Master 127.24.76.126:33541 requested a full tablet report, sending...
I20260808 11:38:26.121009 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 4575
I20260808 11:38:26.121191 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance
I20260808 11:38:26.121886 24881 external_mini_cluster.cc:1428] Setting key bb7302ea57dc7c8199519bdd37cd5779
I20260808 11:38:26.122552 4462 ts_manager.cc:194] Registered new tserver with Master: 5a6215971eda4de2bbb2fbf88c75d70f (127.24.76.65:43047)
I20260808 11:38:26.124245 4462 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.76.65@KRBTEST.COM'} at 127.24.76.65:57391
WARNING: no policy specified for oryx/127.24.76.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.76.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.66.keytab.
Entry for principal oryx/127.24.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.66.keytab.
I20260808 11:38:26.177176 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.66:0
--local_ip_for_outbound_sockets=127.24.76.66
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:33541
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.66.keytab
--principal=oryx/127.24.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:38147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:38:26.511889 4796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:26.512178 4796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:26.512313 4796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:26.519373 4796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:26.519486 4796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:26.519604 4796 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 11:38:26.519713 4796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.66
I20260808 11:38:26.529152 4796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:38147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.66.keytab
--principal=oryx/127.24.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.76.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.76.126:33541
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.66
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:26.530406 4796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:26.531685 4796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:26.544080 4807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:26.544056 4812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:26.544116 4809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:26.544910 4796 server_base.cc:1034] running on GCE node
I20260808 11:38:26.545498 4796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:26.546401 4796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:26.547600 4796 hybrid_clock.cc:648] HybridClock initialized: now 1786189106547554 us; error 79 us; skew 500 ppm
I20260808 11:38:26.547868 4796 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:38:26 dist-test-slave-tc2s krb5kdc[4388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189106, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:26.550917 4796 init.cc:377] Logged in from keytab as oryx/127.24.76.66@KRBTEST.COM (short username oryx)
I20260808 11:38:26.552251 4796 webserver.cc:460] Webserver started at http://127.24.76.66:33107/ using document root <none> and password file <none>
I20260808 11:38:26.552979 4796 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:26.553074 4796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:26.553378 4796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:26.556255 4796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-1/data/instance:
uuid: "b6f5a3d8e29947948e3f0ed47f43402d"
format_stamp: "Formatted at 2026-08-08 11:38:26 on dist-test-slave-tc2s"
server_key: "82542f457efa34a76e64dccfa753be9a"
server_key_iv: "da462c9f3b7fe946c755a472646c72e8"
server_key_version: "encryptionkey@0"
I20260808 11:38:26.557020 4796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance:
uuid: "b6f5a3d8e29947948e3f0ed47f43402d"
format_stamp: "Formatted at 2026-08-08 11:38:26 on dist-test-slave-tc2s"
server_key: "82542f457efa34a76e64dccfa753be9a"
server_key_iv: "da462c9f3b7fe946c755a472646c72e8"
server_key_version: "encryptionkey@0"
I20260808 11:38:26.562059 4796 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 11:38:26.565286 4821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:26.566596 4796 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:38:26.566743 4796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-1/wal
uuid: "b6f5a3d8e29947948e3f0ed47f43402d"
format_stamp: "Formatted at 2026-08-08 11:38:26 on dist-test-slave-tc2s"
server_key: "82542f457efa34a76e64dccfa753be9a"
server_key_iv: "da462c9f3b7fe946c755a472646c72e8"
server_key_version: "encryptionkey@0"
I20260808 11:38:26.566949 4796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:26.588738 4796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:26.592986 4796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:26.593236 4796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:26.595850 4796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:38:26.595955 4796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:26.596109 4796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:38:26.596177 4796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 11:38:26 dist-test-slave-tc2s krb5kdc[4388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189106, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM
I20260808 11:38:26.616117 4610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:26.596189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:52639 (local address 127.24.76.126:33541)
0808 11:38:26.596397 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:26.596404 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:26.597733 (+ 1329us) server_negotiation.cc:407] Connection header received
0808 11:38:26.599794 (+ 2061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:26.599800 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:26.599883 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:26.599973 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:26.601647 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:26.602528 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:26.605049 (+ 2521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:26.605621 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:26.605756 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:26.610565 (+ 4809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:26.610570 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:26.610574 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:26.610608 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:26.612949 (+ 2341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:26.613637 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:26.613643 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:26.613645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:26.613801 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:26.614236 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:26.614241 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:26.614245 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:26.614607 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:26.614741 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:26.615535 (+ 794us) server_negotiation.cc:299] Negotiation successful
0808 11:38:26.615763 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 11:38:26.617075 4839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:26.596684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33541 (local address 127.24.76.66:52639)
0808 11:38:26.597538 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:26.597593 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:26.599518 (+ 1925us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:26.600221 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:26.600231 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:26.600927 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:26.601469 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:26.601494 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:26.602688 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:26.602692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:26.604887 (+ 2195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:26.604903 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:26.605959 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:26.605968 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:26.606313 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:26.607111 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:26.607141 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:26.610361 (+ 3220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:26.613079 (+ 2718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:26.613088 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:26.613103 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:26.613523 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:26.613924 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:26.613929 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:26.613933 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:26.614129 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:26.614740 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:26.614749 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:26.615093 (+ 344us) client_negotiation.cc:769] Sending connection context
0808 11:38:26.615321 (+ 228us) client_negotiation.cc:240] Negotiation successful
0808 11:38:26.615854 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":352,"threads_started":1}
I20260808 11:38:26.639019 4610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:26.624021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:39491 (local address 127.24.76.126:33541)
0808 11:38:26.624186 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:26.624191 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:26.626760 (+ 2569us) server_negotiation.cc:407] Connection header received
0808 11:38:26.626913 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:26.626919 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:26.626978 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:26.627068 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:26.627936 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:26.628632 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:26.629870 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:26.630347 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:26.630481 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:26.632753 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:26.632758 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:26.632761 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:26.632790 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:26.635926 (+ 3136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:26.636642 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:26.636648 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:26.636651 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:26.636812 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:26.637239 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:26.637245 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:26.637248 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:26.637721 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:26.637880 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:26.638479 (+ 599us) server_negotiation.cc:299] Negotiation successful
0808 11:38:26.638690 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 11:38:26.639119 4916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:26.623613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33541 (local address 127.24.76.66:39491)
0808 11:38:26.626669 (+ 3056us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:26.626684 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:26.626804 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:26.627107 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:26.627111 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:26.627533 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:26.627804 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:26.627813 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:26.628771 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:26.628775 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:26.629741 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:26.629752 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:26.630589 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:26.630597 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:26.630822 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:26.631516 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:26.631540 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:26.632576 (+ 1036us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:26.636052 (+ 3476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:26.636058 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:26.636062 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:26.636467 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:26.636922 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:26.636928 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:26.636932 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:26.637110 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:26.637872 (+ 762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:26.637881 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:26.638201 (+ 320us) client_negotiation.cc:769] Sending connection context
0808 11:38:26.638360 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 11:38:26.638680 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2893,"thread_start_us":2704,"threads_started":1}
I20260808 11:38:26.654824 4796 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.66:42771
I20260808 11:38:26.654865 5008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.66:42771 every 8 connection(s)
I20260808 11:38:26.656064 4796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
I20260808 11:38:26.657975 5009 heartbeater.cc:347] Connected to a master server at 127.24.76.126:33541
I20260808 11:38:26.658179 5009 heartbeater.cc:464] Registering TS with master...
I20260808 11:38:26.658787 5009 heartbeater.cc:511] Master 127.24.76.126:33541 requested a full tablet report, sending...
I20260808 11:38:26.660024 4462 ts_manager.cc:194] Registered new tserver with Master: b6f5a3d8e29947948e3f0ed47f43402d (127.24.76.66:42771)
I20260808 11:38:26.660854 4462 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.76.66@KRBTEST.COM'} at 127.24.76.66:52639
I20260808 11:38:26.662024 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 4796
I20260808 11:38:26.662206 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance
I20260808 11:38:26.662779 24881 external_mini_cluster.cc:1428] Setting key a87e056f54d01e8d444ef6e58d7994b0
WARNING: no policy specified for oryx/127.24.76.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.76.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.67.keytab.
Entry for principal oryx/127.24.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.67.keytab.
I20260808 11:38:26.718273 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.67:0
--local_ip_for_outbound_sockets=127.24.76.67
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:33541
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.67.keytab
--principal=oryx/127.24.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:38147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:38:27.040689 5015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:27.040957 5015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:27.041088 5015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:27.048166 5015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:27.048269 5015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:27.048391 5015 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 11:38:27.048492 5015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.67
I20260808 11:38:27.058035 5015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:38147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.67.keytab
--principal=oryx/127.24.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.76.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.76.126:33541
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.67
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:27.059234 5015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:27.060529 5015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:27.072724 5020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:27.072872 5021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:27.072896 5023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:27.073762 5015 server_base.cc:1034] running on GCE node
I20260808 11:38:27.074340 5015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:27.075158 5015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:27.076367 5015 hybrid_clock.cc:648] HybridClock initialized: now 1786189107076303 us; error 40 us; skew 500 ppm
I20260808 11:38:27.076637 5015 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:38:27 dist-test-slave-tc2s krb5kdc[4388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189107, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:27.079589 5015 init.cc:377] Logged in from keytab as oryx/127.24.76.67@KRBTEST.COM (short username oryx)
I20260808 11:38:27.080960 5015 webserver.cc:460] Webserver started at http://127.24.76.67:39951/ using document root <none> and password file <none>
I20260808 11:38:27.081660 5015 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:27.081760 5015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:27.082108 5015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:27.084830 5015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-2/data/instance:
uuid: "ab0c3e1d88d14a1290d8abdf88f84d10"
format_stamp: "Formatted at 2026-08-08 11:38:27 on dist-test-slave-tc2s"
server_key: "68cbefe1bebe27ef843f5b65a038ca94"
server_key_iv: "7fec2bd22ecf150364c3ae93ab368fe7"
server_key_version: "encryptionkey@0"
I20260808 11:38:27.085620 5015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance:
uuid: "ab0c3e1d88d14a1290d8abdf88f84d10"
format_stamp: "Formatted at 2026-08-08 11:38:27 on dist-test-slave-tc2s"
server_key: "68cbefe1bebe27ef843f5b65a038ca94"
server_key_iv: "7fec2bd22ecf150364c3ae93ab368fe7"
server_key_version: "encryptionkey@0"
I20260808 11:38:27.090564 5015 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260808 11:38:27.093751 5032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:27.094969 5015 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 11:38:27.095108 5015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-2/wal
uuid: "ab0c3e1d88d14a1290d8abdf88f84d10"
format_stamp: "Formatted at 2026-08-08 11:38:27 on dist-test-slave-tc2s"
server_key: "68cbefe1bebe27ef843f5b65a038ca94"
server_key_iv: "7fec2bd22ecf150364c3ae93ab368fe7"
server_key_version: "encryptionkey@0"
I20260808 11:38:27.095295 5015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:27.117077 5015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:27.122051 5015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:27.122327 5015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:27.124955 5015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:38:27.125056 5015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:27.125211 5015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:38:27.125331 5015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260808 11:38:27.126809 4789 heartbeater.cc:503] Master 127.24.76.126:33541 was elected leader, sending a full tablet report...
Aug 08 11:38:27 dist-test-slave-tc2s krb5kdc[4388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189107, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM
I20260808 11:38:27.148438 4610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:27.125451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:54633 (local address 127.24.76.126:33541)
0808 11:38:27.125628 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:27.125634 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:27.126953 (+ 1319us) server_negotiation.cc:407] Connection header received
0808 11:38:27.129456 (+ 2503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:27.129462 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:27.129540 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:27.129622 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:27.132811 (+ 3189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:27.133988 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:27.136236 (+ 2248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:27.136788 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:27.136911 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:27.142062 (+ 5151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:27.142069 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:27.142074 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:27.142110 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:27.144976 (+ 2866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:27.145881 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:27.145885 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:27.145887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:27.146041 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:27.146721 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:27.146724 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:27.146727 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:27.147013 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:27.147161 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:27.147850 (+ 689us) server_negotiation.cc:299] Negotiation successful
0808 11:38:27.148082 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 11:38:27.149315 5043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:27.125962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33541 (local address 127.24.76.67:54633)
0808 11:38:27.126762 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:27.126835 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:27.129211 (+ 2376us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:27.129880 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:27.129929 (+ 49us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:27.130586 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:27.132556 (+ 1970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:27.132580 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:27.134170 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:27.134176 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:27.136046 (+ 1870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:27.136066 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:27.137036 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:27.137045 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:27.137237 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:27.138036 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:27.138071 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:27.141729 (+ 3658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:27.145114 (+ 3385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:27.145124 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:27.145140 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:27.145741 (+ 601us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:27.146162 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:27.146168 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:27.146172 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:27.146547 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:27.147150 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:27.147157 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:27.147475 (+ 318us) client_negotiation.cc:769] Sending connection context
0808 11:38:27.147696 (+ 221us) client_negotiation.cc:240] Negotiation successful
0808 11:38:27.148170 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":303,"threads_started":1}
I20260808 11:38:27.167304 5132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:27.154674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33541 (local address 127.24.76.67:34395)
0808 11:38:27.155357 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:27.155377 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:27.155514 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:27.155790 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:27.155795 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:27.156394 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:27.156771 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:27.156783 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:27.157735 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:27.157739 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:27.158814 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:27.158827 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:27.159517 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:27.159523 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:27.159733 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:27.160231 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:27.160247 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:27.160947 (+ 700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:27.164193 (+ 3246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:27.164201 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:27.164205 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:27.164631 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:27.165128 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:27.165133 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:27.165137 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:27.165320 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:27.166108 (+ 788us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:27.166111 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:27.166336 (+ 225us) client_negotiation.cc:769] Sending connection context
0808 11:38:27.166546 (+ 210us) client_negotiation.cc:240] Negotiation successful
0808 11:38:27.166848 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":376,"threads_started":1}
I20260808 11:38:27.168077 4610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:27.154648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:34395 (local address 127.24.76.126:33541)
0808 11:38:27.154826 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:27.154832 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:27.155479 (+ 647us) server_negotiation.cc:407] Connection header received
0808 11:38:27.155615 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:27.155618 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:27.155675 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:27.155751 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:27.156904 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:27.157590 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:27.158949 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:27.159339 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:27.159450 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:27.161124 (+ 1674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:27.161128 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:27.161132 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:27.161159 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:27.164060 (+ 2901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:27.164756 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:27.164762 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:27.164766 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:27.164928 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:27.165504 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:27.165517 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:27.165519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:27.165869 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:27.166045 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:27.167621 (+ 1576us) server_negotiation.cc:299] Negotiation successful
0808 11:38:27.167829 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 11:38:27.182562 5015 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.67:41839
I20260808 11:38:27.182724 5217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.67:41839 every 8 connection(s)
I20260808 11:38:27.183781 5015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
I20260808 11:38:27.185626 5220 heartbeater.cc:347] Connected to a master server at 127.24.76.126:33541
I20260808 11:38:27.185830 5220 heartbeater.cc:464] Registering TS with master...
I20260808 11:38:27.186432 5220 heartbeater.cc:511] Master 127.24.76.126:33541 requested a full tablet report, sending...
I20260808 11:38:27.187758 4462 ts_manager.cc:194] Registered new tserver with Master: ab0c3e1d88d14a1290d8abdf88f84d10 (127.24.76.67:41839)
I20260808 11:38:27.188560 4462 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.76.67@KRBTEST.COM'} at 127.24.76.67:54633
I20260808 11:38:27.191093 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 5015
I20260808 11:38:27.191234 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance
I20260808 11:38:27.191644 24881 external_mini_cluster.cc:1428] Setting key 42e1c5cb94940dc5ae15714f8a12e0be
I20260808 11:38:27.194198 24881 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 11:38:27 dist-test-slave-tc2s krb5kdc[4388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189107, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 11:38:27 dist-test-slave-tc2s krb5kdc[4388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:27 dist-test-slave-tc2s krb5kdc[4388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:27.663596 5009 heartbeater.cc:503] Master 127.24.76.126:33541 was elected leader, sending a full tablet report...
I20260808 11:38:28.190734 5220 heartbeater.cc:503] Master 127.24.76.126:33541 was elected leader, sending a full tablet report...
W20260808 11:38:28.238461 5230 sasl_common.cc:323] Time spent calling sasl_client_start(): real 0.992s user 0.009s sys 0.009s
W20260808 11:38:28.239681 4610 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:27.240882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46334 (local address 127.24.76.126:33541)
0808 11:38:27.241091 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:27.241099 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:27.241312 (+ 213us) server_negotiation.cc:407] Connection header received
0808 11:38:27.241518 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:27.241522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:27.241590 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:27.241675 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:27.243024 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:27.243784 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:27.245028 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:27.245445 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:27.245565 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:28.239344 (+993779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46334: BlockingRecv error: recv got EOF from 127.0.0.1:46334 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 08 11:38:28 dist-test-slave-tc2s krb5kdc[4388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189107, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM
I20260808 11:38:28.255231 4610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:28.242994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46348 (local address 127.24.76.126:33541)
0808 11:38:28.243214 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:28.243221 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:28.243339 (+ 118us) server_negotiation.cc:407] Connection header received
0808 11:38:28.243491 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:28.243495 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:28.243571 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:28.243655 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:28.244792 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.245525 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.246760 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.247160 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.247247 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:28.250769 (+ 3522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:28.250792 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:28.250795 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:28.250829 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:28.253158 (+ 2329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:28.253711 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:28.253714 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:28.253717 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:28.253820 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:28.254124 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:28.254127 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:28.254129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:28.254384 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:28.254572 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:28.254788 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 11:38:28.254967 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 11:38:28.259073 4462 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46348:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:38:28.266883 4462 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:38:28.288110 5252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:28.277925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:41839 (local address 127.0.0.1:33770)
0808 11:38:28.278819 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:28.278856 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:28.279039 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:28.279988 (+ 949us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:28.279994 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:28.280018 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:38:28.280367 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:28.280376 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.282158 (+ 1782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.282165 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:28.284783 (+ 2618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:28.284801 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.287080 (+ 2279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.287088 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:28.287298 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:28.287352 (+ 54us) client_negotiation.cc:769] Sending connection context
0808 11:38:28.287498 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 11:38:28.287687 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"thread_start_us":3064,"threads_started":1}
I20260808 11:38:28.289013 5253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:28.278680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33770 (local address 127.24.76.67:41839)
0808 11:38:28.279239 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:28.279247 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:28.279274 (+ 27us) server_negotiation.cc:407] Connection header received
0808 11:38:28.279350 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:28.279356 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:28.279565 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:28.279760 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:38:28.280597 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.281935 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.284989 (+ 3054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.286585 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.286732 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:28.286943 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:28.288522 (+ 1579us) server_negotiation.cc:299] Negotiation successful
0808 11:38:28.288711 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":798,"threads_started":1}
I20260808 11:38:28.292680 5251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:28.278180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:43047 (local address 127.0.0.1:60252)
0808 11:38:28.281237 (+ 3057us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:28.281253 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:28.281368 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:28.282330 (+ 962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:28.282336 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:28.282363 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:38:28.282856 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:28.282867 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.285657 (+ 2790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.285663 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:28.289273 (+ 3610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:28.289290 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.291930 (+ 2640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.291936 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:28.292128 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:28.292160 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 11:38:28.292254 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 11:38:28.292375 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2938,"thread_start_us":1540,"threads_started":1}
I20260808 11:38:28.292742 5254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:28.279295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60252 (local address 127.24.76.65:43047)
0808 11:38:28.279886 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:28.279893 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:28.281515 (+ 1622us) server_negotiation.cc:407] Connection header received
0808 11:38:28.281635 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:28.281643 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:28.281866 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:28.282081 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:38:28.283054 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.285476 (+ 2422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.289482 (+ 4006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.291476 (+ 1994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.291614 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:28.291798 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:28.292304 (+ 506us) server_negotiation.cc:299] Negotiation successful
0808 11:38:28.292462 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":1165,"threads_started":1}
I20260808 11:38:28.293975 5107 tablet_service.cc:1467] Processing CreateTablet for tablet 5f5d20b870314a198a8d569e30102949 (DEFAULT_TABLE table=test-table [id=50ccb6bbb86b49f49ed38ce2996e943b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:38:28.295284 5107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f5d20b870314a198a8d569e30102949. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:28.296200 4673 tablet_service.cc:1467] Processing CreateTablet for tablet 5f5d20b870314a198a8d569e30102949 (DEFAULT_TABLE table=test-table [id=50ccb6bbb86b49f49ed38ce2996e943b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:38:28.297539 4673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f5d20b870314a198a8d569e30102949. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:28.297542 5252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:28.281396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:42771 (local address 127.0.0.1:43306)
0808 11:38:28.289417 (+ 8021us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:28.289436 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:28.289546 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:28.290971 (+ 1425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:28.290976 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:28.291008 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:38:28.291151 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:28.291162 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.292798 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.292803 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:28.295055 (+ 2252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:28.295071 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.296838 (+ 1767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.296844 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:28.297023 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:28.297055 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 11:38:28.297144 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 11:38:28.297277 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7922,"thread_start_us":8618,"threads_started":1}
I20260808 11:38:28.297654 5256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:28.281665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43306 (local address 127.24.76.66:42771)
0808 11:38:28.290537 (+ 8872us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:28.290543 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:28.290566 (+ 23us) server_negotiation.cc:407] Connection header received
0808 11:38:28.290638 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:28.290643 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:28.290801 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:28.290970 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:38:28.291292 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.292617 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.295227 (+ 2610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.296651 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.296787 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:28.296999 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:28.297220 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 11:38:28.297379 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8731,"thread_start_us":8612,"threads_started":1}
I20260808 11:38:28.300931 4906 tablet_service.cc:1467] Processing CreateTablet for tablet 5f5d20b870314a198a8d569e30102949 (DEFAULT_TABLE table=test-table [id=50ccb6bbb86b49f49ed38ce2996e943b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:38:28.302242 4906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f5d20b870314a198a8d569e30102949. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:28.308225 5257 tablet_bootstrap.cc:492] T 5f5d20b870314a198a8d569e30102949 P 5a6215971eda4de2bbb2fbf88c75d70f: Bootstrap starting.
I20260808 11:38:28.310322 5258 tablet_bootstrap.cc:492] T 5f5d20b870314a198a8d569e30102949 P ab0c3e1d88d14a1290d8abdf88f84d10: Bootstrap starting.
I20260808 11:38:28.312912 5257 tablet_bootstrap.cc:654] T 5f5d20b870314a198a8d569e30102949 P 5a6215971eda4de2bbb2fbf88c75d70f: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:28.313625 5258 tablet_bootstrap.cc:654] T 5f5d20b870314a198a8d569e30102949 P ab0c3e1d88d14a1290d8abdf88f84d10: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:28.314206 5257 log.cc:824] T 5f5d20b870314a198a8d569e30102949 P 5a6215971eda4de2bbb2fbf88c75d70f: Log is configured to *not* fsync() on all Append() calls
I20260808 11:38:28.314674 5259 tablet_bootstrap.cc:492] T 5f5d20b870314a198a8d569e30102949 P b6f5a3d8e29947948e3f0ed47f43402d: Bootstrap starting.
I20260808 11:38:28.314908 5258 log.cc:824] T 5f5d20b870314a198a8d569e30102949 P ab0c3e1d88d14a1290d8abdf88f84d10: Log is configured to *not* fsync() on all Append() calls
I20260808 11:38:28.317451 5259 tablet_bootstrap.cc:654] T 5f5d20b870314a198a8d569e30102949 P b6f5a3d8e29947948e3f0ed47f43402d: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:28.318506 5259 log.cc:824] T 5f5d20b870314a198a8d569e30102949 P b6f5a3d8e29947948e3f0ed47f43402d: Log is configured to *not* fsync() on all Append() calls
I20260808 11:38:28.323578 5258 tablet_bootstrap.cc:492] T 5f5d20b870314a198a8d569e30102949 P ab0c3e1d88d14a1290d8abdf88f84d10: No bootstrap required, opened a new log
I20260808 11:38:28.323819 5258 ts_tablet_manager.cc:1397] T 5f5d20b870314a198a8d569e30102949 P ab0c3e1d88d14a1290d8abdf88f84d10: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.008s
I20260808 11:38:28.323913 5257 tablet_bootstrap.cc:492] T 5f5d20b870314a198a8d569e30102949 P 5a6215971eda4de2bbb2fbf88c75d70f: No bootstrap required, opened a new log
I20260808 11:38:28.323951 5259 tablet_bootstrap.cc:492] T 5f5d20b870314a198a8d569e30102949 P b6f5a3d8e29947948e3f0ed47f43402d: No bootstrap required, opened a new log
I20260808 11:38:28.324182 5257 ts_tablet_manager.cc:1397] T 5f5d20b870314a198a8d569e30102949 P 5a6215971eda4de2bbb2fbf88c75d70f: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.006s
I20260808 11:38:28.324182 5259 ts_tablet_manager.cc:1397] T 5f5d20b870314a198a8d569e30102949 P b6f5a3d8e29947948e3f0ed47f43402d: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260808 11:38:28.331099 5258 raft_consensus.cc:348] T 5f5d20b870314a198a8d569e30102949 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } } peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } }
I20260808 11:38:28.331437 5258 raft_consensus.cc:374] T 5f5d20b870314a198a8d569e30102949 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:28.331543 5258 raft_consensus.cc:729] T 5f5d20b870314a198a8d569e30102949 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab0c3e1d88d14a1290d8abdf88f84d10, State: Initialized, Role: FOLLOWER
I20260808 11:38:28.332086 5258 consensus_queue.cc:260] T 5f5d20b870314a198a8d569e30102949 P ab0c3e1d88d14a1290d8abdf88f84d10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } } peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } }
I20260808 11:38:28.333696 5258 ts_tablet_manager.cc:1428] T 5f5d20b870314a198a8d569e30102949 P ab0c3e1d88d14a1290d8abdf88f84d10: Time spent starting tablet: real 0.010s user 0.001s sys 0.010s
I20260808 11:38:28.334244 5257 raft_consensus.cc:348] T 5f5d20b870314a198a8d569e30102949 P 5a6215971eda4de2bbb2fbf88c75d70f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } } peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } }
I20260808 11:38:28.334244 5259 raft_consensus.cc:348] T 5f5d20b870314a198a8d569e30102949 P b6f5a3d8e29947948e3f0ed47f43402d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } } peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } }
I20260808 11:38:28.334638 5259 raft_consensus.cc:374] T 5f5d20b870314a198a8d569e30102949 P b6f5a3d8e29947948e3f0ed47f43402d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:28.334638 5257 raft_consensus.cc:374] T 5f5d20b870314a198a8d569e30102949 P 5a6215971eda4de2bbb2fbf88c75d70f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:28.334740 5259 raft_consensus.cc:729] T 5f5d20b870314a198a8d569e30102949 P b6f5a3d8e29947948e3f0ed47f43402d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6f5a3d8e29947948e3f0ed47f43402d, State: Initialized, Role: FOLLOWER
I20260808 11:38:28.334740 5257 raft_consensus.cc:729] T 5f5d20b870314a198a8d569e30102949 P 5a6215971eda4de2bbb2fbf88c75d70f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a6215971eda4de2bbb2fbf88c75d70f, State: Initialized, Role: FOLLOWER
I20260808 11:38:28.335307 5259 consensus_queue.cc:260] T 5f5d20b870314a198a8d569e30102949 P b6f5a3d8e29947948e3f0ed47f43402d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } } peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } }
I20260808 11:38:28.335350 5257 consensus_queue.cc:260] T 5f5d20b870314a198a8d569e30102949 P 5a6215971eda4de2bbb2fbf88c75d70f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } } peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } }
I20260808 11:38:28.337047 5257 ts_tablet_manager.cc:1428] T 5f5d20b870314a198a8d569e30102949 P 5a6215971eda4de2bbb2fbf88c75d70f: Time spent starting tablet: real 0.013s user 0.004s sys 0.009s
I20260808 11:38:28.338541 5259 ts_tablet_manager.cc:1428] T 5f5d20b870314a198a8d569e30102949 P b6f5a3d8e29947948e3f0ed47f43402d: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260808 11:38:28.367547 5271 raft_consensus.cc:482] T 5f5d20b870314a198a8d569e30102949 P b6f5a3d8e29947948e3f0ed47f43402d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:38:28.367784 5271 raft_consensus.cc:504] T 5f5d20b870314a198a8d569e30102949 P b6f5a3d8e29947948e3f0ed47f43402d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } } peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } }
I20260808 11:38:28.369292 5271 leader_election.cc:288] T 5f5d20b870314a198a8d569e30102949 P b6f5a3d8e29947948e3f0ed47f43402d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ab0c3e1d88d14a1290d8abdf88f84d10 (127.24.76.67:41839), 5a6215971eda4de2bbb2fbf88c75d70f (127.24.76.65:43047)
W20260808 11:38:28.370738 4790 tablet.cc:2369] T 5f5d20b870314a198a8d569e30102949 P 5a6215971eda4de2bbb2fbf88c75d70f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:38:28.376145 5254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:28.369759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:52119 (local address 127.24.76.65:43047)
0808 11:38:28.370010 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:28.370016 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:28.370769 (+ 753us) server_negotiation.cc:407] Connection header received
0808 11:38:28.370983 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:28.370989 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:28.371059 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:28.371124 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:38:28.372011 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.372714 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.374517 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.375262 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.375403 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:28.375477 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:28.375816 (+ 339us) server_negotiation.cc:299] Negotiation successful
0808 11:38:28.375933 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 11:38:28.376144 5253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:28.369600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:38493 (local address 127.24.76.67:41839)
0808 11:38:28.369795 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:28.369801 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:28.370647 (+ 846us) server_negotiation.cc:407] Connection header received
0808 11:38:28.370907 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:28.370912 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:28.370995 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:28.371046 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:38:28.371832 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.372581 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.374525 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.375272 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.375413 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:28.375483 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:28.375768 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 11:38:28.375897 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 11:38:28.376497 5276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:28.369743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:43047 (local address 127.24.76.66:52119)
0808 11:38:28.370664 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:28.370682 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:28.370816 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:28.371321 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:28.371326 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:28.371364 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:38:28.371830 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:28.371841 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.372848 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.372852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:28.374376 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:28.374388 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.375554 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.375558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:28.375667 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:28.375688 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 11:38:28.375777 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 11:38:28.375895 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":784,"mutex_wait_us":21,"thread_start_us":509,"threads_started":1}
I20260808 11:38:28.376753 5275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:28.369667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:41839 (local address 127.24.76.66:38493)
0808 11:38:28.370547 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:28.370562 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:28.370695 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:28.371232 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:28.371237 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:28.371263 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:38:28.371679 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:28.371690 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.372713 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.372718 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:28.374371 (+ 1653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:28.374383 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.375501 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.375509 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:28.375628 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:28.375650 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 11:38:28.375720 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 11:38:28.375831 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":759,"mutex_wait_us":97,"thread_start_us":551,"threads_started":1}
I20260808 11:38:28.377516 4702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f5d20b870314a198a8d569e30102949" candidate_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" is_pre_election: true
I20260808 11:38:28.377614 5148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f5d20b870314a198a8d569e30102949" candidate_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" is_pre_election: true
I20260808 11:38:28.378005 4702 raft_consensus.cc:2454] T 5f5d20b870314a198a8d569e30102949 P 5a6215971eda4de2bbb2fbf88c75d70f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6f5a3d8e29947948e3f0ed47f43402d in term 0.
I20260808 11:38:28.378080 5148 raft_consensus.cc:2454] T 5f5d20b870314a198a8d569e30102949 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6f5a3d8e29947948e3f0ed47f43402d in term 0.
I20260808 11:38:28.378564 4828 leader_election.cc:302] T 5f5d20b870314a198a8d569e30102949 P b6f5a3d8e29947948e3f0ed47f43402d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a6215971eda4de2bbb2fbf88c75d70f, b6f5a3d8e29947948e3f0ed47f43402d; no voters:
I20260808 11:38:28.378885 5271 raft_consensus.cc:2781] T 5f5d20b870314a198a8d569e30102949 P b6f5a3d8e29947948e3f0ed47f43402d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:38:28.379025 5271 raft_consensus.cc:482] T 5f5d20b870314a198a8d569e30102949 P b6f5a3d8e29947948e3f0ed47f43402d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:38:28.379125 5271 raft_consensus.cc:3037] T 5f5d20b870314a198a8d569e30102949 P b6f5a3d8e29947948e3f0ed47f43402d [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:28.381246 5271 raft_consensus.cc:504] T 5f5d20b870314a198a8d569e30102949 P b6f5a3d8e29947948e3f0ed47f43402d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } } peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } }
I20260808 11:38:28.381794 5271 leader_election.cc:288] T 5f5d20b870314a198a8d569e30102949 P b6f5a3d8e29947948e3f0ed47f43402d [CANDIDATE]: Term 1 election: Requested vote from peers ab0c3e1d88d14a1290d8abdf88f84d10 (127.24.76.67:41839), 5a6215971eda4de2bbb2fbf88c75d70f (127.24.76.65:43047)
I20260808 11:38:28.382184 5148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f5d20b870314a198a8d569e30102949" candidate_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10"
I20260808 11:38:28.382247 4702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f5d20b870314a198a8d569e30102949" candidate_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a6215971eda4de2bbb2fbf88c75d70f"
I20260808 11:38:28.382393 5148 raft_consensus.cc:3037] T 5f5d20b870314a198a8d569e30102949 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:28.382427 4702 raft_consensus.cc:3037] T 5f5d20b870314a198a8d569e30102949 P 5a6215971eda4de2bbb2fbf88c75d70f [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:28.385458 5148 raft_consensus.cc:2454] T 5f5d20b870314a198a8d569e30102949 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6f5a3d8e29947948e3f0ed47f43402d in term 1.
I20260808 11:38:28.385458 4702 raft_consensus.cc:2454] T 5f5d20b870314a198a8d569e30102949 P 5a6215971eda4de2bbb2fbf88c75d70f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6f5a3d8e29947948e3f0ed47f43402d in term 1.
I20260808 11:38:28.385830 4828 leader_election.cc:302] T 5f5d20b870314a198a8d569e30102949 P b6f5a3d8e29947948e3f0ed47f43402d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5a6215971eda4de2bbb2fbf88c75d70f, ab0c3e1d88d14a1290d8abdf88f84d10, b6f5a3d8e29947948e3f0ed47f43402d; no voters:
I20260808 11:38:28.386184 5271 raft_consensus.cc:2781] T 5f5d20b870314a198a8d569e30102949 P b6f5a3d8e29947948e3f0ed47f43402d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:38:28.386580 5271 raft_consensus.cc:686] T 5f5d20b870314a198a8d569e30102949 P b6f5a3d8e29947948e3f0ed47f43402d [term 1 LEADER]: Becoming Leader. State: Replica: b6f5a3d8e29947948e3f0ed47f43402d, State: Running, Role: LEADER
I20260808 11:38:28.387058 5271 consensus_queue.cc:237] T 5f5d20b870314a198a8d569e30102949 P b6f5a3d8e29947948e3f0ed47f43402d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } } peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } }
I20260808 11:38:28.391427 4461 catalog_manager.cc:5495] T 5f5d20b870314a198a8d569e30102949 P b6f5a3d8e29947948e3f0ed47f43402d reported cstate change: term changed from 0 to 1, leader changed from <none> to b6f5a3d8e29947948e3f0ed47f43402d (127.24.76.66). New cstate: current_term: 1 leader_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } health_report { overall_health: HEALTHY } } }
W20260808 11:38:28.408620 5011 tablet.cc:2369] T 5f5d20b870314a198a8d569e30102949 P b6f5a3d8e29947948e3f0ed47f43402d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:38:28.412062 5256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:28.407120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43316 (local address 127.24.76.66:42771)
0808 11:38:28.407274 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:28.407278 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:28.407457 (+ 179us) server_negotiation.cc:407] Connection header received
0808 11:38:28.407640 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:28.407643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:28.407721 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:28.407831 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:38:28.408121 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.409059 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.410337 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.410738 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.410828 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:28.411172 (+ 344us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:38:28.411655 (+ 483us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:38:28.411757 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:28.411811 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 11:38:28.411875 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 11:38:28.419853 5148 raft_consensus.cc:1261] T 5f5d20b870314a198a8d569e30102949 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 1 FOLLOWER]: Refusing update from remote peer b6f5a3d8e29947948e3f0ed47f43402d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:38:28.419852 4702 raft_consensus.cc:1261] T 5f5d20b870314a198a8d569e30102949 P 5a6215971eda4de2bbb2fbf88c75d70f [term 1 FOLLOWER]: Refusing update from remote peer b6f5a3d8e29947948e3f0ed47f43402d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:38:28.420872 5271 consensus_queue.cc:1035] T 5f5d20b870314a198a8d569e30102949 P b6f5a3d8e29947948e3f0ed47f43402d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:38:28.421157 5278 consensus_queue.cc:1035] T 5f5d20b870314a198a8d569e30102949 P b6f5a3d8e29947948e3f0ed47f43402d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:38:28.448796 4461 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:38:28.449074 4461 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:38:28.453253 4461 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 508408a84d5c4b838a2198a913aff5a2: Sending DeleteTablet for 3 replicas of tablet 5f5d20b870314a198a8d569e30102949
I20260808 11:38:28.454500 5107 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f5d20b870314a198a8d569e30102949 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:38:28 UTC) from {username='oryx', principal='oryx/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:33770
I20260808 11:38:28.454492 4673 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f5d20b870314a198a8d569e30102949 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:38:28 UTC) from {username='oryx', principal='oryx/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:60252
I20260808 11:38:28.455191 4906 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f5d20b870314a198a8d569e30102949 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:38:28 UTC) from {username='oryx', principal='oryx/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:43306
I20260808 11:38:28.460036 5300 tablet_replica.cc:330] T 5f5d20b870314a198a8d569e30102949 P b6f5a3d8e29947948e3f0ed47f43402d: stopping tablet replica
I20260808 11:38:28.460624 5300 raft_consensus.cc:2229] T 5f5d20b870314a198a8d569e30102949 P b6f5a3d8e29947948e3f0ed47f43402d [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:38:28.461299 5300 raft_consensus.cc:2258] T 5f5d20b870314a198a8d569e30102949 P b6f5a3d8e29947948e3f0ed47f43402d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:38:28.461597 4461 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46348:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:38:28.462693 4461 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:38:28.470393 5298 tablet_replica.cc:330] T 5f5d20b870314a198a8d569e30102949 P 5a6215971eda4de2bbb2fbf88c75d70f: stopping tablet replica
I20260808 11:38:28.470989 5298 raft_consensus.cc:2229] T 5f5d20b870314a198a8d569e30102949 P 5a6215971eda4de2bbb2fbf88c75d70f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:38:28.471025 5105 tablet_service.cc:1467] Processing CreateTablet for tablet 9cac91f050244fc3beb910945bd44db8 (DEFAULT_TABLE table=test-table [id=d95da4a34e1b45dd8fe444710ecd3b97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:38:28.471621 5105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cac91f050244fc3beb910945bd44db8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:28.471468 4906 tablet_service.cc:1467] Processing CreateTablet for tablet 9cac91f050244fc3beb910945bd44db8 (DEFAULT_TABLE table=test-table [id=d95da4a34e1b45dd8fe444710ecd3b97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:38:28.471618 5298 raft_consensus.cc:2258] T 5f5d20b870314a198a8d569e30102949 P 5a6215971eda4de2bbb2fbf88c75d70f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:38:28.472002 4906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cac91f050244fc3beb910945bd44db8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:28.472615 5299 tablet_replica.cc:330] T 5f5d20b870314a198a8d569e30102949 P ab0c3e1d88d14a1290d8abdf88f84d10: stopping tablet replica
I20260808 11:38:28.473057 5299 raft_consensus.cc:2229] T 5f5d20b870314a198a8d569e30102949 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:38:28.473598 5299 raft_consensus.cc:2258] T 5f5d20b870314a198a8d569e30102949 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:38:28.474273 4673 tablet_service.cc:1467] Processing CreateTablet for tablet 9cac91f050244fc3beb910945bd44db8 (DEFAULT_TABLE table=test-table [id=d95da4a34e1b45dd8fe444710ecd3b97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:38:28.474941 4673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cac91f050244fc3beb910945bd44db8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:28.478302 5258 tablet_bootstrap.cc:492] T 9cac91f050244fc3beb910945bd44db8 P ab0c3e1d88d14a1290d8abdf88f84d10: Bootstrap starting.
I20260808 11:38:28.476658 5298 ts_tablet_manager.cc:1905] T 5f5d20b870314a198a8d569e30102949 P 5a6215971eda4de2bbb2fbf88c75d70f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:38:28.479825 5258 tablet_bootstrap.cc:654] T 9cac91f050244fc3beb910945bd44db8 P ab0c3e1d88d14a1290d8abdf88f84d10: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:28.482874 5259 tablet_bootstrap.cc:492] T 9cac91f050244fc3beb910945bd44db8 P b6f5a3d8e29947948e3f0ed47f43402d: Bootstrap starting.
I20260808 11:38:28.484653 5259 tablet_bootstrap.cc:654] T 9cac91f050244fc3beb910945bd44db8 P b6f5a3d8e29947948e3f0ed47f43402d: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:28.485834 5300 ts_tablet_manager.cc:1905] T 5f5d20b870314a198a8d569e30102949 P b6f5a3d8e29947948e3f0ed47f43402d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:38:28.487509 5299 ts_tablet_manager.cc:1905] T 5f5d20b870314a198a8d569e30102949 P ab0c3e1d88d14a1290d8abdf88f84d10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:38:28.491072 5257 tablet_bootstrap.cc:492] T 9cac91f050244fc3beb910945bd44db8 P 5a6215971eda4de2bbb2fbf88c75d70f: Bootstrap starting.
I20260808 11:38:28.492839 5257 tablet_bootstrap.cc:654] T 9cac91f050244fc3beb910945bd44db8 P 5a6215971eda4de2bbb2fbf88c75d70f: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:28.495957 5298 ts_tablet_manager.cc:1918] T 5f5d20b870314a198a8d569e30102949 P 5a6215971eda4de2bbb2fbf88c75d70f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:38:28.497360 5259 tablet_bootstrap.cc:492] T 9cac91f050244fc3beb910945bd44db8 P b6f5a3d8e29947948e3f0ed47f43402d: No bootstrap required, opened a new log
I20260808 11:38:28.497532 5259 ts_tablet_manager.cc:1397] T 9cac91f050244fc3beb910945bd44db8 P b6f5a3d8e29947948e3f0ed47f43402d: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.000s
I20260808 11:38:28.497578 5298 log.cc:1196] T 5f5d20b870314a198a8d569e30102949 P 5a6215971eda4de2bbb2fbf88c75d70f: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-0/wal/wals/5f5d20b870314a198a8d569e30102949
I20260808 11:38:28.498373 5298 ts_tablet_manager.cc:1939] T 5f5d20b870314a198a8d569e30102949 P 5a6215971eda4de2bbb2fbf88c75d70f: Deleting consensus metadata
I20260808 11:38:28.499577 5259 raft_consensus.cc:348] T 9cac91f050244fc3beb910945bd44db8 P b6f5a3d8e29947948e3f0ed47f43402d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } }
I20260808 11:38:28.499846 5259 raft_consensus.cc:374] T 9cac91f050244fc3beb910945bd44db8 P b6f5a3d8e29947948e3f0ed47f43402d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:28.499950 5259 raft_consensus.cc:729] T 9cac91f050244fc3beb910945bd44db8 P b6f5a3d8e29947948e3f0ed47f43402d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6f5a3d8e29947948e3f0ed47f43402d, State: Initialized, Role: FOLLOWER
I20260808 11:38:28.500663 5259 consensus_queue.cc:260] T 9cac91f050244fc3beb910945bd44db8 P b6f5a3d8e29947948e3f0ed47f43402d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } }
I20260808 11:38:28.501359 5259 ts_tablet_manager.cc:1428] T 9cac91f050244fc3beb910945bd44db8 P b6f5a3d8e29947948e3f0ed47f43402d: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260808 11:38:28.502126 4441 catalog_manager.cc:4841] TS 5a6215971eda4de2bbb2fbf88c75d70f (127.24.76.65:43047): tablet 5f5d20b870314a198a8d569e30102949 (table test-table [id=50ccb6bbb86b49f49ed38ce2996e943b]) successfully deleted
I20260808 11:38:28.502759 5258 tablet_bootstrap.cc:492] T 9cac91f050244fc3beb910945bd44db8 P ab0c3e1d88d14a1290d8abdf88f84d10: No bootstrap required, opened a new log
I20260808 11:38:28.502914 5258 ts_tablet_manager.cc:1397] T 9cac91f050244fc3beb910945bd44db8 P ab0c3e1d88d14a1290d8abdf88f84d10: Time spent bootstrapping tablet: real 0.025s user 0.004s sys 0.005s
I20260808 11:38:28.503417 5300 ts_tablet_manager.cc:1918] T 5f5d20b870314a198a8d569e30102949 P b6f5a3d8e29947948e3f0ed47f43402d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:38:28.503664 5300 log.cc:1196] T 5f5d20b870314a198a8d569e30102949 P b6f5a3d8e29947948e3f0ed47f43402d: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-1/wal/wals/5f5d20b870314a198a8d569e30102949
I20260808 11:38:28.503708 5258 raft_consensus.cc:348] T 9cac91f050244fc3beb910945bd44db8 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } }
I20260808 11:38:28.504382 5300 ts_tablet_manager.cc:1939] T 5f5d20b870314a198a8d569e30102949 P b6f5a3d8e29947948e3f0ed47f43402d: Deleting consensus metadata
I20260808 11:38:28.503944 5258 raft_consensus.cc:374] T 9cac91f050244fc3beb910945bd44db8 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:28.504550 5258 raft_consensus.cc:729] T 9cac91f050244fc3beb910945bd44db8 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab0c3e1d88d14a1290d8abdf88f84d10, State: Initialized, Role: FOLLOWER
I20260808 11:38:28.504877 5258 consensus_queue.cc:260] T 9cac91f050244fc3beb910945bd44db8 P ab0c3e1d88d14a1290d8abdf88f84d10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } }
I20260808 11:38:28.506244 4440 catalog_manager.cc:4841] TS b6f5a3d8e29947948e3f0ed47f43402d (127.24.76.66:42771): tablet 5f5d20b870314a198a8d569e30102949 (table test-table [id=50ccb6bbb86b49f49ed38ce2996e943b]) successfully deleted
I20260808 11:38:28.506954 5257 tablet_bootstrap.cc:492] T 9cac91f050244fc3beb910945bd44db8 P 5a6215971eda4de2bbb2fbf88c75d70f: No bootstrap required, opened a new log
I20260808 11:38:28.507108 5257 ts_tablet_manager.cc:1397] T 9cac91f050244fc3beb910945bd44db8 P 5a6215971eda4de2bbb2fbf88c75d70f: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.000s
I20260808 11:38:28.507977 5258 ts_tablet_manager.cc:1428] T 9cac91f050244fc3beb910945bd44db8 P ab0c3e1d88d14a1290d8abdf88f84d10: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260808 11:38:28.507922 5257 raft_consensus.cc:348] T 9cac91f050244fc3beb910945bd44db8 P 5a6215971eda4de2bbb2fbf88c75d70f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } }
I20260808 11:38:28.508140 5257 raft_consensus.cc:374] T 9cac91f050244fc3beb910945bd44db8 P 5a6215971eda4de2bbb2fbf88c75d70f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:28.508241 5257 raft_consensus.cc:729] T 9cac91f050244fc3beb910945bd44db8 P 5a6215971eda4de2bbb2fbf88c75d70f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a6215971eda4de2bbb2fbf88c75d70f, State: Initialized, Role: FOLLOWER
I20260808 11:38:28.508550 5257 consensus_queue.cc:260] T 9cac91f050244fc3beb910945bd44db8 P 5a6215971eda4de2bbb2fbf88c75d70f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } }
I20260808 11:38:28.510773 5257 ts_tablet_manager.cc:1428] T 9cac91f050244fc3beb910945bd44db8 P 5a6215971eda4de2bbb2fbf88c75d70f: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260808 11:38:28.511102 5299 ts_tablet_manager.cc:1918] T 5f5d20b870314a198a8d569e30102949 P ab0c3e1d88d14a1290d8abdf88f84d10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:38:28.511250 5299 log.cc:1196] T 5f5d20b870314a198a8d569e30102949 P ab0c3e1d88d14a1290d8abdf88f84d10: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-2/wal/wals/5f5d20b870314a198a8d569e30102949
I20260808 11:38:28.511776 5299 ts_tablet_manager.cc:1939] T 5f5d20b870314a198a8d569e30102949 P ab0c3e1d88d14a1290d8abdf88f84d10: Deleting consensus metadata
I20260808 11:38:28.513264 4440 catalog_manager.cc:4841] TS ab0c3e1d88d14a1290d8abdf88f84d10 (127.24.76.67:41839): tablet 5f5d20b870314a198a8d569e30102949 (table test-table [id=50ccb6bbb86b49f49ed38ce2996e943b]) successfully deleted
I20260808 11:38:28.575031 5274 raft_consensus.cc:482] T 9cac91f050244fc3beb910945bd44db8 P 5a6215971eda4de2bbb2fbf88c75d70f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:38:28.575279 5274 raft_consensus.cc:504] T 9cac91f050244fc3beb910945bd44db8 P 5a6215971eda4de2bbb2fbf88c75d70f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } }
I20260808 11:38:28.577270 5274 leader_election.cc:288] T 9cac91f050244fc3beb910945bd44db8 P 5a6215971eda4de2bbb2fbf88c75d70f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6f5a3d8e29947948e3f0ed47f43402d (127.24.76.66:42771), ab0c3e1d88d14a1290d8abdf88f84d10 (127.24.76.67:41839)
I20260808 11:38:28.585484 5253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:28.578385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:37847 (local address 127.24.76.67:41839)
0808 11:38:28.578554 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:28.578560 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:28.579362 (+ 802us) server_negotiation.cc:407] Connection header received
0808 11:38:28.579653 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:28.579659 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:28.579735 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:28.579792 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:38:28.580599 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.581492 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.583210 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.584396 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.584562 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:28.584638 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:28.585061 (+ 423us) server_negotiation.cc:299] Negotiation successful
0808 11:38:28.585187 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 11:38:28.585553 5256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:28.577564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:54289 (local address 127.24.76.66:42771)
0808 11:38:28.577767 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:28.577773 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:28.578182 (+ 409us) server_negotiation.cc:407] Connection header received
0808 11:38:28.578505 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:28.578511 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:28.578595 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:28.578665 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:38:28.579693 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.580682 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.583305 (+ 2623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.584520 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.584662 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:28.584725 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:28.585129 (+ 404us) server_negotiation.cc:299] Negotiation successful
0808 11:38:28.585245 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 11:38:28.585870 5311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:28.578222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:41839 (local address 127.24.76.65:37847)
0808 11:38:28.579253 (+ 1031us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:28.579275 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:28.579485 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:28.579993 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:28.579998 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:28.580023 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:38:28.580415 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:28.580429 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.581618 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.581623 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:28.583054 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:28.583066 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.584617 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.584624 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:28.584878 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:28.584917 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 11:38:28.585014 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 11:38:28.585145 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":901,"mutex_wait_us":29,"thread_start_us":718,"threads_started":1}
I20260808 11:38:28.586333 5307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:28.577496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:42771 (local address 127.24.76.65:54289)
0808 11:38:28.578091 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:28.578109 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:28.578302 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:28.579087 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:28.579092 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:28.579121 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:38:28.579500 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:28.579515 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.580820 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.580826 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:28.583153 (+ 2327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:28.583169 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.584815 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.584820 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:28.584958 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:28.584982 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 11:38:28.585721 (+ 739us) client_negotiation.cc:240] Negotiation successful
0808 11:38:28.585855 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"mutex_wait_us":2,"thread_start_us":301,"threads_started":1}
I20260808 11:38:28.586609 5148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cac91f050244fc3beb910945bd44db8" candidate_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" is_pre_election: true
I20260808 11:38:28.586871 5148 raft_consensus.cc:2454] T 9cac91f050244fc3beb910945bd44db8 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a6215971eda4de2bbb2fbf88c75d70f in term 0.
I20260808 11:38:28.587406 4603 leader_election.cc:302] T 9cac91f050244fc3beb910945bd44db8 P 5a6215971eda4de2bbb2fbf88c75d70f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a6215971eda4de2bbb2fbf88c75d70f, ab0c3e1d88d14a1290d8abdf88f84d10; no voters:
I20260808 11:38:28.587342 4936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cac91f050244fc3beb910945bd44db8" candidate_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" is_pre_election: true
I20260808 11:38:28.587754 4936 raft_consensus.cc:2454] T 9cac91f050244fc3beb910945bd44db8 P b6f5a3d8e29947948e3f0ed47f43402d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a6215971eda4de2bbb2fbf88c75d70f in term 0.
I20260808 11:38:28.587823 5274 raft_consensus.cc:2781] T 9cac91f050244fc3beb910945bd44db8 P 5a6215971eda4de2bbb2fbf88c75d70f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:38:28.587944 5274 raft_consensus.cc:482] T 9cac91f050244fc3beb910945bd44db8 P 5a6215971eda4de2bbb2fbf88c75d70f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:38:28.588038 5274 raft_consensus.cc:3037] T 9cac91f050244fc3beb910945bd44db8 P 5a6215971eda4de2bbb2fbf88c75d70f [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:28.591257 5274 raft_consensus.cc:504] T 9cac91f050244fc3beb910945bd44db8 P 5a6215971eda4de2bbb2fbf88c75d70f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } }
I20260808 11:38:28.592003 5274 leader_election.cc:288] T 9cac91f050244fc3beb910945bd44db8 P 5a6215971eda4de2bbb2fbf88c75d70f [CANDIDATE]: Term 1 election: Requested vote from peers b6f5a3d8e29947948e3f0ed47f43402d (127.24.76.66:42771), ab0c3e1d88d14a1290d8abdf88f84d10 (127.24.76.67:41839)
I20260808 11:38:28.592399 4936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cac91f050244fc3beb910945bd44db8" candidate_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6f5a3d8e29947948e3f0ed47f43402d"
I20260808 11:38:28.592419 5148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cac91f050244fc3beb910945bd44db8" candidate_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10"
I20260808 11:38:28.592593 4936 raft_consensus.cc:3037] T 9cac91f050244fc3beb910945bd44db8 P b6f5a3d8e29947948e3f0ed47f43402d [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:28.592614 5148 raft_consensus.cc:3037] T 9cac91f050244fc3beb910945bd44db8 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:28.594684 5148 raft_consensus.cc:2454] T 9cac91f050244fc3beb910945bd44db8 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a6215971eda4de2bbb2fbf88c75d70f in term 1.
I20260808 11:38:28.595022 4603 leader_election.cc:302] T 9cac91f050244fc3beb910945bd44db8 P 5a6215971eda4de2bbb2fbf88c75d70f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a6215971eda4de2bbb2fbf88c75d70f, ab0c3e1d88d14a1290d8abdf88f84d10; no voters:
I20260808 11:38:28.595258 5274 raft_consensus.cc:2781] T 9cac91f050244fc3beb910945bd44db8 P 5a6215971eda4de2bbb2fbf88c75d70f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:38:28.595700 5274 raft_consensus.cc:686] T 9cac91f050244fc3beb910945bd44db8 P 5a6215971eda4de2bbb2fbf88c75d70f [term 1 LEADER]: Becoming Leader. State: Replica: 5a6215971eda4de2bbb2fbf88c75d70f, State: Running, Role: LEADER
I20260808 11:38:28.595827 4936 raft_consensus.cc:2454] T 9cac91f050244fc3beb910945bd44db8 P b6f5a3d8e29947948e3f0ed47f43402d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a6215971eda4de2bbb2fbf88c75d70f in term 1.
I20260808 11:38:28.596361 5274 consensus_queue.cc:237] T 9cac91f050244fc3beb910945bd44db8 P 5a6215971eda4de2bbb2fbf88c75d70f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } }
I20260808 11:38:28.599732 4462 catalog_manager.cc:5495] T 9cac91f050244fc3beb910945bd44db8 P 5a6215971eda4de2bbb2fbf88c75d70f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a6215971eda4de2bbb2fbf88c75d70f (127.24.76.65). New cstate: current_term: 1 leader_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } health_report { overall_health: UNKNOWN } } }
Aug 08 11:38:28 dist-test-slave-tc2s krb5kdc[4388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189105, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.126@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM
I20260808 11:38:28.642580 4610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:28.628089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46354 (local address 127.24.76.126:33541)
0808 11:38:28.628331 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:28.628338 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:28.628811 (+ 473us) server_negotiation.cc:407] Connection header received
0808 11:38:28.628942 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:28.628948 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:28.629023 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:28.629129 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:28.630253 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.631447 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.632657 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.633290 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.633411 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:28.636708 (+ 3297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:28.636715 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:28.636720 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:28.636755 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:28.639937 (+ 3182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:28.640612 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:28.640617 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:28.640621 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:28.640771 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:28.641126 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:28.641129 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:28.641131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:28.641382 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:28.641514 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:28.642024 (+ 510us) server_negotiation.cc:299] Negotiation successful
0808 11:38:28.642282 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":92,"server-negotiator.queue_time_us":107}
I20260808 11:38:28.642635 5320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:28.628103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33541 (local address 127.0.0.1:46354)
0808 11:38:28.628711 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:28.628726 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:28.628838 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:28.629144 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:28.629148 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:28.629793 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:28.630112 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:28.630122 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.631590 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.631594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:28.632522 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:28.632533 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.633446 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.633452 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:28.633571 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:28.634244 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:28.634260 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:28.636523 (+ 2263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:28.640076 (+ 3553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:28.640086 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:28.640092 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:28.640505 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:28.640869 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:28.640872 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:28.640874 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:28.641019 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:28.641510 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:28.641518 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:28.641699 (+ 181us) client_negotiation.cc:769] Sending connection context
0808 11:38:28.641881 (+ 182us) client_negotiation.cc:240] Negotiation successful
0808 11:38:28.642174 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":336,"threads_started":1}
I20260808 11:38:28.647696 4462 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:46354:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 11:38:28.648669 4462 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:38:28.655382 4673 tablet_service.cc:1467] Processing CreateTablet for tablet b6e70ff9d1674e7c981f582694164be3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8c748192da3f43af9d8ae387472d72de]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 11:38:28.655967 4673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6e70ff9d1674e7c981f582694164be3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:28.661712 4906 tablet_service.cc:1467] Processing CreateTablet for tablet b6e70ff9d1674e7c981f582694164be3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8c748192da3f43af9d8ae387472d72de]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 11:38:28.661707 5107 tablet_service.cc:1467] Processing CreateTablet for tablet b6e70ff9d1674e7c981f582694164be3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8c748192da3f43af9d8ae387472d72de]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 11:38:28.662292 4906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6e70ff9d1674e7c981f582694164be3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:28.662302 5107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6e70ff9d1674e7c981f582694164be3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:28.663074 5257 tablet_bootstrap.cc:492] T b6e70ff9d1674e7c981f582694164be3 P 5a6215971eda4de2bbb2fbf88c75d70f: Bootstrap starting.
I20260808 11:38:28.664601 5257 tablet_bootstrap.cc:654] T b6e70ff9d1674e7c981f582694164be3 P 5a6215971eda4de2bbb2fbf88c75d70f: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:28.670363 5258 tablet_bootstrap.cc:492] T b6e70ff9d1674e7c981f582694164be3 P ab0c3e1d88d14a1290d8abdf88f84d10: Bootstrap starting.
I20260808 11:38:28.670480 5259 tablet_bootstrap.cc:492] T b6e70ff9d1674e7c981f582694164be3 P b6f5a3d8e29947948e3f0ed47f43402d: Bootstrap starting.
I20260808 11:38:28.671976 5258 tablet_bootstrap.cc:654] T b6e70ff9d1674e7c981f582694164be3 P ab0c3e1d88d14a1290d8abdf88f84d10: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:28.672140 5259 tablet_bootstrap.cc:654] T b6e70ff9d1674e7c981f582694164be3 P b6f5a3d8e29947948e3f0ed47f43402d: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:28.684386 5258 tablet_bootstrap.cc:492] T b6e70ff9d1674e7c981f582694164be3 P ab0c3e1d88d14a1290d8abdf88f84d10: No bootstrap required, opened a new log
I20260808 11:38:28.684715 5258 ts_tablet_manager.cc:1397] T b6e70ff9d1674e7c981f582694164be3 P ab0c3e1d88d14a1290d8abdf88f84d10: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.006s
I20260808 11:38:28.685494 5258 raft_consensus.cc:348] T b6e70ff9d1674e7c981f582694164be3 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } }
I20260808 11:38:28.685843 5258 raft_consensus.cc:374] T b6e70ff9d1674e7c981f582694164be3 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:28.686064 5258 raft_consensus.cc:729] T b6e70ff9d1674e7c981f582694164be3 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab0c3e1d88d14a1290d8abdf88f84d10, State: Initialized, Role: FOLLOWER
I20260808 11:38:28.686431 5258 consensus_queue.cc:260] T b6e70ff9d1674e7c981f582694164be3 P ab0c3e1d88d14a1290d8abdf88f84d10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } }
W20260808 11:38:28.686913 5221 tablet.cc:2369] T 9cac91f050244fc3beb910945bd44db8 P ab0c3e1d88d14a1290d8abdf88f84d10: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:38:28.687502 5270 tablet_replica.cc:439] T b6e70ff9d1674e7c981f582694164be3 P ab0c3e1d88d14a1290d8abdf88f84d10: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } } }
I20260808 11:38:28.687862 5270 tablet_replica.cc:442] T b6e70ff9d1674e7c981f582694164be3 P ab0c3e1d88d14a1290d8abdf88f84d10: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:38:28.688145 5259 tablet_bootstrap.cc:492] T b6e70ff9d1674e7c981f582694164be3 P b6f5a3d8e29947948e3f0ed47f43402d: No bootstrap required, opened a new log
I20260808 11:38:28.688318 5259 ts_tablet_manager.cc:1397] T b6e70ff9d1674e7c981f582694164be3 P b6f5a3d8e29947948e3f0ed47f43402d: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.008s
I20260808 11:38:28.689038 5258 ts_tablet_manager.cc:1428] T b6e70ff9d1674e7c981f582694164be3 P ab0c3e1d88d14a1290d8abdf88f84d10: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260808 11:38:28.689163 5259 raft_consensus.cc:348] T b6e70ff9d1674e7c981f582694164be3 P b6f5a3d8e29947948e3f0ed47f43402d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } }
I20260808 11:38:28.689417 5259 raft_consensus.cc:374] T b6e70ff9d1674e7c981f582694164be3 P b6f5a3d8e29947948e3f0ed47f43402d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:28.689508 5259 raft_consensus.cc:729] T b6e70ff9d1674e7c981f582694164be3 P b6f5a3d8e29947948e3f0ed47f43402d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6f5a3d8e29947948e3f0ed47f43402d, State: Initialized, Role: FOLLOWER
I20260808 11:38:28.689746 5259 consensus_queue.cc:260] T b6e70ff9d1674e7c981f582694164be3 P b6f5a3d8e29947948e3f0ed47f43402d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } }
I20260808 11:38:28.690178 5271 tablet_replica.cc:439] T b6e70ff9d1674e7c981f582694164be3 P b6f5a3d8e29947948e3f0ed47f43402d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } } }
I20260808 11:38:28.690490 5271 tablet_replica.cc:442] T b6e70ff9d1674e7c981f582694164be3 P b6f5a3d8e29947948e3f0ed47f43402d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:38:28.691108 5259 ts_tablet_manager.cc:1428] T b6e70ff9d1674e7c981f582694164be3 P b6f5a3d8e29947948e3f0ed47f43402d: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260808 11:38:28.699543 5257 tablet_bootstrap.cc:492] T b6e70ff9d1674e7c981f582694164be3 P 5a6215971eda4de2bbb2fbf88c75d70f: No bootstrap required, opened a new log
I20260808 11:38:28.699716 5257 ts_tablet_manager.cc:1397] T b6e70ff9d1674e7c981f582694164be3 P 5a6215971eda4de2bbb2fbf88c75d70f: Time spent bootstrapping tablet: real 0.037s user 0.006s sys 0.025s
I20260808 11:38:28.700464 5257 raft_consensus.cc:348] T b6e70ff9d1674e7c981f582694164be3 P 5a6215971eda4de2bbb2fbf88c75d70f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } }
I20260808 11:38:28.700703 5257 raft_consensus.cc:374] T b6e70ff9d1674e7c981f582694164be3 P 5a6215971eda4de2bbb2fbf88c75d70f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:28.700768 5257 raft_consensus.cc:729] T b6e70ff9d1674e7c981f582694164be3 P 5a6215971eda4de2bbb2fbf88c75d70f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a6215971eda4de2bbb2fbf88c75d70f, State: Initialized, Role: FOLLOWER
I20260808 11:38:28.701020 5257 consensus_queue.cc:260] T b6e70ff9d1674e7c981f582694164be3 P 5a6215971eda4de2bbb2fbf88c75d70f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } }
I20260808 11:38:28.701402 5274 tablet_replica.cc:439] T b6e70ff9d1674e7c981f582694164be3 P 5a6215971eda4de2bbb2fbf88c75d70f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } } }
I20260808 11:38:28.701675 5274 tablet_replica.cc:442] T b6e70ff9d1674e7c981f582694164be3 P 5a6215971eda4de2bbb2fbf88c75d70f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:38:28.702566 5257 ts_tablet_manager.cc:1428] T b6e70ff9d1674e7c981f582694164be3 P 5a6215971eda4de2bbb2fbf88c75d70f: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260808 11:38:28.800349 5271 raft_consensus.cc:482] T b6e70ff9d1674e7c981f582694164be3 P b6f5a3d8e29947948e3f0ed47f43402d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:38:28.800544 5271 raft_consensus.cc:504] T b6e70ff9d1674e7c981f582694164be3 P b6f5a3d8e29947948e3f0ed47f43402d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } }
I20260808 11:38:28.801209 5271 leader_election.cc:288] T b6e70ff9d1674e7c981f582694164be3 P b6f5a3d8e29947948e3f0ed47f43402d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a6215971eda4de2bbb2fbf88c75d70f (127.24.76.65:43047), ab0c3e1d88d14a1290d8abdf88f84d10 (127.24.76.67:41839)
I20260808 11:38:28.801755 5148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6e70ff9d1674e7c981f582694164be3" candidate_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" is_pre_election: true
I20260808 11:38:28.801808 4702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6e70ff9d1674e7c981f582694164be3" candidate_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" is_pre_election: true
I20260808 11:38:28.802037 5148 raft_consensus.cc:2454] T b6e70ff9d1674e7c981f582694164be3 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6f5a3d8e29947948e3f0ed47f43402d in term 0.
I20260808 11:38:28.802134 4702 raft_consensus.cc:2454] T b6e70ff9d1674e7c981f582694164be3 P 5a6215971eda4de2bbb2fbf88c75d70f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6f5a3d8e29947948e3f0ed47f43402d in term 0.
I20260808 11:38:28.802506 4827 leader_election.cc:302] T b6e70ff9d1674e7c981f582694164be3 P b6f5a3d8e29947948e3f0ed47f43402d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab0c3e1d88d14a1290d8abdf88f84d10, b6f5a3d8e29947948e3f0ed47f43402d; no voters:
I20260808 11:38:28.802850 5271 raft_consensus.cc:2781] T b6e70ff9d1674e7c981f582694164be3 P b6f5a3d8e29947948e3f0ed47f43402d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:38:28.802987 5271 raft_consensus.cc:482] T b6e70ff9d1674e7c981f582694164be3 P b6f5a3d8e29947948e3f0ed47f43402d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:38:28.803097 5271 raft_consensus.cc:3037] T b6e70ff9d1674e7c981f582694164be3 P b6f5a3d8e29947948e3f0ed47f43402d [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:28.806382 5271 raft_consensus.cc:504] T b6e70ff9d1674e7c981f582694164be3 P b6f5a3d8e29947948e3f0ed47f43402d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } }
I20260808 11:38:28.807109 5271 leader_election.cc:288] T b6e70ff9d1674e7c981f582694164be3 P b6f5a3d8e29947948e3f0ed47f43402d [CANDIDATE]: Term 1 election: Requested vote from peers 5a6215971eda4de2bbb2fbf88c75d70f (127.24.76.65:43047), ab0c3e1d88d14a1290d8abdf88f84d10 (127.24.76.67:41839)
I20260808 11:38:28.807555 5148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6e70ff9d1674e7c981f582694164be3" candidate_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10"
I20260808 11:38:28.807608 4702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6e70ff9d1674e7c981f582694164be3" candidate_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a6215971eda4de2bbb2fbf88c75d70f"
I20260808 11:38:28.807778 5148 raft_consensus.cc:3037] T b6e70ff9d1674e7c981f582694164be3 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:28.807837 4702 raft_consensus.cc:3037] T b6e70ff9d1674e7c981f582694164be3 P 5a6215971eda4de2bbb2fbf88c75d70f [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:28.809827 5148 raft_consensus.cc:2454] T b6e70ff9d1674e7c981f582694164be3 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6f5a3d8e29947948e3f0ed47f43402d in term 1.
I20260808 11:38:28.810227 4827 leader_election.cc:302] T b6e70ff9d1674e7c981f582694164be3 P b6f5a3d8e29947948e3f0ed47f43402d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab0c3e1d88d14a1290d8abdf88f84d10, b6f5a3d8e29947948e3f0ed47f43402d; no voters:
I20260808 11:38:28.810493 5271 raft_consensus.cc:2781] T b6e70ff9d1674e7c981f582694164be3 P b6f5a3d8e29947948e3f0ed47f43402d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:38:28.810660 5271 raft_consensus.cc:686] T b6e70ff9d1674e7c981f582694164be3 P b6f5a3d8e29947948e3f0ed47f43402d [term 1 LEADER]: Becoming Leader. State: Replica: b6f5a3d8e29947948e3f0ed47f43402d, State: Running, Role: LEADER
I20260808 11:38:28.810761 4702 raft_consensus.cc:2454] T b6e70ff9d1674e7c981f582694164be3 P 5a6215971eda4de2bbb2fbf88c75d70f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6f5a3d8e29947948e3f0ed47f43402d in term 1.
I20260808 11:38:28.810949 5271 consensus_queue.cc:237] T b6e70ff9d1674e7c981f582694164be3 P b6f5a3d8e29947948e3f0ed47f43402d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } }
I20260808 11:38:28.812788 5278 tablet_replica.cc:439] T b6e70ff9d1674e7c981f582694164be3 P b6f5a3d8e29947948e3f0ed47f43402d: TxnStatusTablet state changed. Reason: New leader b6f5a3d8e29947948e3f0ed47f43402d. Latest consensus state: current_term: 1 leader_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } } }
I20260808 11:38:28.813153 5278 tablet_replica.cc:442] T b6e70ff9d1674e7c981f582694164be3 P b6f5a3d8e29947948e3f0ed47f43402d: This TxnStatusTablet replica's current role is: LEADER
I20260808 11:38:28.814401 4462 catalog_manager.cc:5495] T b6e70ff9d1674e7c981f582694164be3 P b6f5a3d8e29947948e3f0ed47f43402d reported cstate change: term changed from 0 to 1, leader changed from <none> to b6f5a3d8e29947948e3f0ed47f43402d (127.24.76.66). New cstate: current_term: 1 leader_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } health_report { overall_health: UNKNOWN } } }
I20260808 11:38:28.817248 5327 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 11:38:28.817518 5327 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 11:38:28.861229 5320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:28.856146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:42771 (local address 127.0.0.1:43318)
0808 11:38:28.856396 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:28.856417 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:28.856568 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:28.856990 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:28.856995 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:28.857007 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 11:38:28.857189 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:28.857199 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.858313 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.858319 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:28.859380 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:28.859393 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.860128 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.860134 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:28.860310 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:28.860367 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 11:38:28.860742 (+ 375us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 11:38:28.860773 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 11:38:28.860850 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 11:38:28.861001 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 11:38:28.861307 5256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:28.856227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43318 (local address 127.24.76.66:42771)
0808 11:38:28.856409 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:28.856416 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:28.856480 (+ 64us) server_negotiation.cc:407] Connection header received
0808 11:38:28.856784 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:28.856788 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:28.856858 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:28.856940 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:38:28.857322 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.858180 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.859522 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.859973 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.860087 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:28.860542 (+ 455us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:38:28.860648 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:38:28.860746 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:28.860914 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 11:38:28.861015 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 11:38:28.864933 4702 raft_consensus.cc:1261] T b6e70ff9d1674e7c981f582694164be3 P 5a6215971eda4de2bbb2fbf88c75d70f [term 1 FOLLOWER]: Refusing update from remote peer b6f5a3d8e29947948e3f0ed47f43402d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:38:28.864961 5148 raft_consensus.cc:1261] T b6e70ff9d1674e7c981f582694164be3 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 1 FOLLOWER]: Refusing update from remote peer b6f5a3d8e29947948e3f0ed47f43402d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:38:28.865635 5278 consensus_queue.cc:1035] T b6e70ff9d1674e7c981f582694164be3 P b6f5a3d8e29947948e3f0ed47f43402d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:38:28.865953 5271 consensus_queue.cc:1035] T b6e70ff9d1674e7c981f582694164be3 P b6f5a3d8e29947948e3f0ed47f43402d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:38:28.868954 5274 tablet_replica.cc:439] T b6e70ff9d1674e7c981f582694164be3 P 5a6215971eda4de2bbb2fbf88c75d70f: TxnStatusTablet state changed. Reason: New leader b6f5a3d8e29947948e3f0ed47f43402d. Latest consensus state: current_term: 1 leader_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } } }
I20260808 11:38:28.869268 5274 tablet_replica.cc:442] T b6e70ff9d1674e7c981f582694164be3 P 5a6215971eda4de2bbb2fbf88c75d70f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:38:28.872207 5285 mvcc.cc:205] Tried to move back new op lower bound from 7316230589904728064 to 7316230589692395520. Current Snapshot: MvccSnapshot[applied={T|T < 7316230589904728064}]
I20260808 11:38:28.872442 5274 tablet_replica.cc:439] T b6e70ff9d1674e7c981f582694164be3 P 5a6215971eda4de2bbb2fbf88c75d70f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } } }
I20260808 11:38:28.872702 5274 tablet_replica.cc:442] T b6e70ff9d1674e7c981f582694164be3 P 5a6215971eda4de2bbb2fbf88c75d70f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:38:28.871387 5278 tablet_replica.cc:439] T b6e70ff9d1674e7c981f582694164be3 P b6f5a3d8e29947948e3f0ed47f43402d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } } }
I20260808 11:38:28.873600 5278 tablet_replica.cc:442] T b6e70ff9d1674e7c981f582694164be3 P b6f5a3d8e29947948e3f0ed47f43402d: This TxnStatusTablet replica's current role is: LEADER
I20260808 11:38:28.872108 5283 mvcc.cc:205] Tried to move back new op lower bound from 7316230589904728064 to 7316230589692395520. Current Snapshot: MvccSnapshot[applied={T|T < 7316230589904728064 or (T in {7316230589904728064})}]
I20260808 11:38:28.875319 5270 tablet_replica.cc:439] T b6e70ff9d1674e7c981f582694164be3 P ab0c3e1d88d14a1290d8abdf88f84d10: TxnStatusTablet state changed. Reason: New leader b6f5a3d8e29947948e3f0ed47f43402d. Latest consensus state: current_term: 1 leader_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } } }
I20260808 11:38:28.875599 5270 tablet_replica.cc:442] T b6e70ff9d1674e7c981f582694164be3 P ab0c3e1d88d14a1290d8abdf88f84d10: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:38:28.878274 5271 tablet_replica.cc:439] T b6e70ff9d1674e7c981f582694164be3 P b6f5a3d8e29947948e3f0ed47f43402d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } } }
I20260808 11:38:28.878496 5271 tablet_replica.cc:442] T b6e70ff9d1674e7c981f582694164be3 P b6f5a3d8e29947948e3f0ed47f43402d: This TxnStatusTablet replica's current role is: LEADER
I20260808 11:38:28.879953 5286 mvcc.cc:205] Tried to move back new op lower bound from 7316230589904728064 to 7316230589692395520. Current Snapshot: MvccSnapshot[applied={T|T < 7316230589904728064}]
I20260808 11:38:28.880663 5270 tablet_replica.cc:439] T b6e70ff9d1674e7c981f582694164be3 P ab0c3e1d88d14a1290d8abdf88f84d10: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } } }
I20260808 11:38:28.880875 5270 tablet_replica.cc:442] T b6e70ff9d1674e7c981f582694164be3 P ab0c3e1d88d14a1290d8abdf88f84d10: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 11:38:28.887683 5333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:28.881387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60258 (local address 127.24.76.65:43047)
0808 11:38:28.881844 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:28.881849 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:28.881877 (+ 28us) server_negotiation.cc:407] Connection header received
0808 11:38:28.881985 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:28.881990 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:28.882078 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:28.882245 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:38:28.882720 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.883859 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.885195 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.885828 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.886000 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:28.886464 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:38:28.887107 (+ 643us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:38:28.887257 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:28.887300 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 11:38:28.887382 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":254,"threads_started":1}
I20260808 11:38:28.902848 5337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:28.897284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:42771 (local address 127.24.76.65:39109)
0808 11:38:28.897971 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:28.897988 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:28.898098 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:28.898374 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:28.898377 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:28.898386 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 11:38:28.898638 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:28.898646 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.899869 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.899874 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:28.900961 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:28.900973 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.901877 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.901881 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:28.902031 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:28.902082 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 11:38:28.902425 (+ 343us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 11:38:28.902432 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 11:38:28.902486 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 11:38:28.902576 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":362,"threads_started":1}
I20260808 11:38:28.902884 5256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:28.897298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:39109 (local address 127.24.76.66:42771)
0808 11:38:28.897464 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:28.897470 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:28.898060 (+ 590us) server_negotiation.cc:407] Connection header received
0808 11:38:28.898200 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:28.898206 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:28.898281 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:28.898362 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:38:28.898761 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.899750 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.901099 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.901728 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.901834 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:28.902228 (+ 394us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:38:28.902346 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:38:28.902429 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:28.902512 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 11:38:28.902587 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 11:38:28.912051 5148 raft_consensus.cc:1261] T 9cac91f050244fc3beb910945bd44db8 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 1 FOLLOWER]: Refusing update from remote peer 5a6215971eda4de2bbb2fbf88c75d70f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:38:28.912317 4936 raft_consensus.cc:1261] T 9cac91f050244fc3beb910945bd44db8 P b6f5a3d8e29947948e3f0ed47f43402d [term 1 FOLLOWER]: Refusing update from remote peer 5a6215971eda4de2bbb2fbf88c75d70f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:38:28.912684 5274 consensus_queue.cc:1035] T 9cac91f050244fc3beb910945bd44db8 P 5a6215971eda4de2bbb2fbf88c75d70f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:38:28.912983 5312 consensus_queue.cc:1035] T 9cac91f050244fc3beb910945bd44db8 P 5a6215971eda4de2bbb2fbf88c75d70f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:38:28.954420 5333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:28.948068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:56251 (local address 127.24.76.65:43047)
0808 11:38:28.948232 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:28.948238 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:28.948565 (+ 327us) server_negotiation.cc:407] Connection header received
0808 11:38:28.948717 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:28.948722 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:28.948803 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:28.948905 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:38:28.949310 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.950422 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.952504 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.952948 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.953069 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:28.953639 (+ 570us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:38:28.953756 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:38:28.953877 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:28.954040 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 11:38:28.954120 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 11:38:28.954553 5346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:28.947657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:43047 (local address 127.24.76.66:56251)
0808 11:38:28.948473 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:28.948489 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:28.948604 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:28.948914 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:28.948917 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:28.948927 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 11:38:28.949171 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:28.949178 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.950582 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.950588 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:28.952343 (+ 1755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:28.952359 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:28.953191 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:28.953198 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:28.953394 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:28.953467 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 11:38:28.953880 (+ 413us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 11:38:28.953917 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 11:38:28.954006 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 11:38:28.954143 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":485,"threads_started":1}
I20260808 11:38:28.999750 4461 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:38:29.000025 4461 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:38:29.003612 4461 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 508408a84d5c4b838a2198a913aff5a2: Sending DeleteTablet for 3 replicas of tablet 9cac91f050244fc3beb910945bd44db8
I20260808 11:38:29.004314 4673 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cac91f050244fc3beb910945bd44db8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:38:28 UTC) from {username='oryx', principal='oryx/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:60252
I20260808 11:38:29.004410 4906 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cac91f050244fc3beb910945bd44db8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:38:28 UTC) from {username='oryx', principal='oryx/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:43306
I20260808 11:38:29.004704 5300 tablet_replica.cc:330] T 9cac91f050244fc3beb910945bd44db8 P b6f5a3d8e29947948e3f0ed47f43402d: stopping tablet replica
I20260808 11:38:29.004915 5300 raft_consensus.cc:2229] T 9cac91f050244fc3beb910945bd44db8 P b6f5a3d8e29947948e3f0ed47f43402d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:38:29.005117 5300 raft_consensus.cc:2258] T 9cac91f050244fc3beb910945bd44db8 P b6f5a3d8e29947948e3f0ed47f43402d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:38:29.005968 5107 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cac91f050244fc3beb910945bd44db8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:38:28 UTC) from {username='oryx', principal='oryx/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:33770
I20260808 11:38:29.006225 5300 ts_tablet_manager.cc:1905] T 9cac91f050244fc3beb910945bd44db8 P b6f5a3d8e29947948e3f0ed47f43402d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:38:29.006242 5299 tablet_replica.cc:330] T 9cac91f050244fc3beb910945bd44db8 P ab0c3e1d88d14a1290d8abdf88f84d10: stopping tablet replica
I20260808 11:38:29.006438 5299 raft_consensus.cc:2229] T 9cac91f050244fc3beb910945bd44db8 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:38:29.006640 5299 raft_consensus.cc:2258] T 9cac91f050244fc3beb910945bd44db8 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:38:29.007753 5299 ts_tablet_manager.cc:1905] T 9cac91f050244fc3beb910945bd44db8 P ab0c3e1d88d14a1290d8abdf88f84d10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:38:29.015086 5299 ts_tablet_manager.cc:1918] T 9cac91f050244fc3beb910945bd44db8 P ab0c3e1d88d14a1290d8abdf88f84d10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 11:38:29.015236 5299 log.cc:1196] T 9cac91f050244fc3beb910945bd44db8 P ab0c3e1d88d14a1290d8abdf88f84d10: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-2/wal/wals/9cac91f050244fc3beb910945bd44db8
I20260808 11:38:29.019441 5300 ts_tablet_manager.cc:1918] T 9cac91f050244fc3beb910945bd44db8 P b6f5a3d8e29947948e3f0ed47f43402d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 11:38:29.020179 5300 log.cc:1196] T 9cac91f050244fc3beb910945bd44db8 P b6f5a3d8e29947948e3f0ed47f43402d: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-1/wal/wals/9cac91f050244fc3beb910945bd44db8
I20260808 11:38:29.020824 5300 ts_tablet_manager.cc:1939] T 9cac91f050244fc3beb910945bd44db8 P b6f5a3d8e29947948e3f0ed47f43402d: Deleting consensus metadata
I20260808 11:38:29.023048 4440 catalog_manager.cc:4841] TS b6f5a3d8e29947948e3f0ed47f43402d (127.24.76.66:42771): tablet 9cac91f050244fc3beb910945bd44db8 (table test-table [id=d95da4a34e1b45dd8fe444710ecd3b97]) successfully deleted
I20260808 11:38:29.015654 5299 ts_tablet_manager.cc:1939] T 9cac91f050244fc3beb910945bd44db8 P ab0c3e1d88d14a1290d8abdf88f84d10: Deleting consensus metadata
I20260808 11:38:29.027112 4440 catalog_manager.cc:4841] TS ab0c3e1d88d14a1290d8abdf88f84d10 (127.24.76.67:41839): tablet 9cac91f050244fc3beb910945bd44db8 (table test-table [id=d95da4a34e1b45dd8fe444710ecd3b97]) successfully deleted
I20260808 11:38:29.031188 5349 tablet_replica.cc:330] T 9cac91f050244fc3beb910945bd44db8 P 5a6215971eda4de2bbb2fbf88c75d70f: stopping tablet replica
I20260808 11:38:29.031424 5349 raft_consensus.cc:2229] T 9cac91f050244fc3beb910945bd44db8 P 5a6215971eda4de2bbb2fbf88c75d70f [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:38:29.031983 5349 raft_consensus.cc:2258] T 9cac91f050244fc3beb910945bd44db8 P 5a6215971eda4de2bbb2fbf88c75d70f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:38:29.033142 5349 ts_tablet_manager.cc:1905] T 9cac91f050244fc3beb910945bd44db8 P 5a6215971eda4de2bbb2fbf88c75d70f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:38:29.040791 5349 ts_tablet_manager.cc:1918] T 9cac91f050244fc3beb910945bd44db8 P 5a6215971eda4de2bbb2fbf88c75d70f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 11:38:29.040983 5349 log.cc:1196] T 9cac91f050244fc3beb910945bd44db8 P 5a6215971eda4de2bbb2fbf88c75d70f: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-0/wal/wals/9cac91f050244fc3beb910945bd44db8
I20260808 11:38:29.041528 5349 ts_tablet_manager.cc:1939] T 9cac91f050244fc3beb910945bd44db8 P 5a6215971eda4de2bbb2fbf88c75d70f: Deleting consensus metadata
I20260808 11:38:29.042732 4441 catalog_manager.cc:4841] TS 5a6215971eda4de2bbb2fbf88c75d70f (127.24.76.65:43047): tablet 9cac91f050244fc3beb910945bd44db8 (table test-table [id=d95da4a34e1b45dd8fe444710ecd3b97]) successfully deleted
Aug 08 11:38:29 dist-test-slave-tc2s krb5kdc[4388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 11:38:29 dist-test-slave-tc2s krb5kdc[4388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:29 dist-test-slave-tc2s krb5kdc[4388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 11:38:29.099557 4610 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:29.085445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46362 (local address 127.24.76.126:33541)
0808 11:38:29.085646 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:29.085652 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:29.086478 (+ 826us) server_negotiation.cc:407] Connection header received
0808 11:38:29.086566 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:29.086572 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:29.086663 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:29.086767 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:29.088564 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:29.089749 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:29.091760 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:29.092466 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:29.092594 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:29.099227 (+ 6633us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46362: BlockingRecv error: recv got EOF from 127.0.0.1:46362 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 08 11:38:29 dist-test-slave-tc2s krb5kdc[4388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM
I20260808 11:38:29.121026 4610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:29.103528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46378 (local address 127.24.76.126:33541)
0808 11:38:29.103693 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:29.103699 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:29.103718 (+ 19us) server_negotiation.cc:407] Connection header received
0808 11:38:29.103897 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:29.103904 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:29.103984 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:29.104053 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:29.105725 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:29.106871 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:29.108692 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:29.109344 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:29.109468 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:29.114249 (+ 4781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:29.114255 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:29.114260 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:29.114306 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:29.117944 (+ 3638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:29.118774 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:29.118780 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:29.118783 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:29.118924 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:29.119351 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:29.119356 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:29.119359 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:29.119686 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:29.119936 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:29.120471 (+ 535us) server_negotiation.cc:299] Negotiation successful
0808 11:38:29.120671 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 11:38:29.125670 4461 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46378:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:38:29.126449 4461 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:38:29.135032 4906 tablet_service.cc:1467] Processing CreateTablet for tablet 31ffcaf64da44826a308a5f7de96a409 (DEFAULT_TABLE table=test-table [id=f7dcfa1e85034a1c8c630fb993e7e2f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:38:29.135504 4906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31ffcaf64da44826a308a5f7de96a409. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:29.138310 5107 tablet_service.cc:1467] Processing CreateTablet for tablet 31ffcaf64da44826a308a5f7de96a409 (DEFAULT_TABLE table=test-table [id=f7dcfa1e85034a1c8c630fb993e7e2f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:38:29.138758 5107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31ffcaf64da44826a308a5f7de96a409. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:29.142549 4673 tablet_service.cc:1467] Processing CreateTablet for tablet 31ffcaf64da44826a308a5f7de96a409 (DEFAULT_TABLE table=test-table [id=f7dcfa1e85034a1c8c630fb993e7e2f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:38:29.142969 4673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31ffcaf64da44826a308a5f7de96a409. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:29.158317 5258 tablet_bootstrap.cc:492] T 31ffcaf64da44826a308a5f7de96a409 P ab0c3e1d88d14a1290d8abdf88f84d10: Bootstrap starting.
I20260808 11:38:29.158744 5257 tablet_bootstrap.cc:492] T 31ffcaf64da44826a308a5f7de96a409 P 5a6215971eda4de2bbb2fbf88c75d70f: Bootstrap starting.
I20260808 11:38:29.160434 5257 tablet_bootstrap.cc:654] T 31ffcaf64da44826a308a5f7de96a409 P 5a6215971eda4de2bbb2fbf88c75d70f: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:29.160632 5258 tablet_bootstrap.cc:654] T 31ffcaf64da44826a308a5f7de96a409 P ab0c3e1d88d14a1290d8abdf88f84d10: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:29.161432 5259 tablet_bootstrap.cc:492] T 31ffcaf64da44826a308a5f7de96a409 P b6f5a3d8e29947948e3f0ed47f43402d: Bootstrap starting.
I20260808 11:38:29.163138 5259 tablet_bootstrap.cc:654] T 31ffcaf64da44826a308a5f7de96a409 P b6f5a3d8e29947948e3f0ed47f43402d: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:29.174734 5258 tablet_bootstrap.cc:492] T 31ffcaf64da44826a308a5f7de96a409 P ab0c3e1d88d14a1290d8abdf88f84d10: No bootstrap required, opened a new log
I20260808 11:38:29.174968 5258 ts_tablet_manager.cc:1397] T 31ffcaf64da44826a308a5f7de96a409 P ab0c3e1d88d14a1290d8abdf88f84d10: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.006s
I20260808 11:38:29.175830 5258 raft_consensus.cc:348] T 31ffcaf64da44826a308a5f7de96a409 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } }
I20260808 11:38:29.176247 5258 raft_consensus.cc:374] T 31ffcaf64da44826a308a5f7de96a409 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:29.176396 5258 raft_consensus.cc:729] T 31ffcaf64da44826a308a5f7de96a409 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab0c3e1d88d14a1290d8abdf88f84d10, State: Initialized, Role: FOLLOWER
I20260808 11:38:29.176725 5258 consensus_queue.cc:260] T 31ffcaf64da44826a308a5f7de96a409 P ab0c3e1d88d14a1290d8abdf88f84d10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } }
I20260808 11:38:29.178639 5258 ts_tablet_manager.cc:1428] T 31ffcaf64da44826a308a5f7de96a409 P ab0c3e1d88d14a1290d8abdf88f84d10: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260808 11:38:29.181109 5257 tablet_bootstrap.cc:492] T 31ffcaf64da44826a308a5f7de96a409 P 5a6215971eda4de2bbb2fbf88c75d70f: No bootstrap required, opened a new log
I20260808 11:38:29.181270 5257 ts_tablet_manager.cc:1397] T 31ffcaf64da44826a308a5f7de96a409 P 5a6215971eda4de2bbb2fbf88c75d70f: Time spent bootstrapping tablet: real 0.023s user 0.011s sys 0.001s
I20260808 11:38:29.182091 5257 raft_consensus.cc:348] T 31ffcaf64da44826a308a5f7de96a409 P 5a6215971eda4de2bbb2fbf88c75d70f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } }
I20260808 11:38:29.182369 5257 raft_consensus.cc:374] T 31ffcaf64da44826a308a5f7de96a409 P 5a6215971eda4de2bbb2fbf88c75d70f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:29.182476 5257 raft_consensus.cc:729] T 31ffcaf64da44826a308a5f7de96a409 P 5a6215971eda4de2bbb2fbf88c75d70f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a6215971eda4de2bbb2fbf88c75d70f, State: Initialized, Role: FOLLOWER
I20260808 11:38:29.182608 5259 tablet_bootstrap.cc:492] T 31ffcaf64da44826a308a5f7de96a409 P b6f5a3d8e29947948e3f0ed47f43402d: No bootstrap required, opened a new log
I20260808 11:38:29.182803 5259 ts_tablet_manager.cc:1397] T 31ffcaf64da44826a308a5f7de96a409 P b6f5a3d8e29947948e3f0ed47f43402d: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.014s
I20260808 11:38:29.182732 5257 consensus_queue.cc:260] T 31ffcaf64da44826a308a5f7de96a409 P 5a6215971eda4de2bbb2fbf88c75d70f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } }
I20260808 11:38:29.183718 5259 raft_consensus.cc:348] T 31ffcaf64da44826a308a5f7de96a409 P b6f5a3d8e29947948e3f0ed47f43402d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } }
I20260808 11:38:29.184055 5259 raft_consensus.cc:374] T 31ffcaf64da44826a308a5f7de96a409 P b6f5a3d8e29947948e3f0ed47f43402d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:29.184998 5257 ts_tablet_manager.cc:1428] T 31ffcaf64da44826a308a5f7de96a409 P 5a6215971eda4de2bbb2fbf88c75d70f: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260808 11:38:29.187985 5259 raft_consensus.cc:729] T 31ffcaf64da44826a308a5f7de96a409 P b6f5a3d8e29947948e3f0ed47f43402d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6f5a3d8e29947948e3f0ed47f43402d, State: Initialized, Role: FOLLOWER
I20260808 11:38:29.188390 5259 consensus_queue.cc:260] T 31ffcaf64da44826a308a5f7de96a409 P b6f5a3d8e29947948e3f0ed47f43402d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } }
I20260808 11:38:29.189095 5259 ts_tablet_manager.cc:1428] T 31ffcaf64da44826a308a5f7de96a409 P b6f5a3d8e29947948e3f0ed47f43402d: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260808 11:38:29.223261 5341 raft_consensus.cc:482] T 31ffcaf64da44826a308a5f7de96a409 P 5a6215971eda4de2bbb2fbf88c75d70f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:38:29.223534 5341 raft_consensus.cc:504] T 31ffcaf64da44826a308a5f7de96a409 P 5a6215971eda4de2bbb2fbf88c75d70f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } }
I20260808 11:38:29.224383 5341 leader_election.cc:288] T 31ffcaf64da44826a308a5f7de96a409 P 5a6215971eda4de2bbb2fbf88c75d70f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6f5a3d8e29947948e3f0ed47f43402d (127.24.76.66:42771), ab0c3e1d88d14a1290d8abdf88f84d10 (127.24.76.67:41839)
I20260808 11:38:29.225034 4936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31ffcaf64da44826a308a5f7de96a409" candidate_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" is_pre_election: true
I20260808 11:38:29.225291 4936 raft_consensus.cc:2454] T 31ffcaf64da44826a308a5f7de96a409 P b6f5a3d8e29947948e3f0ed47f43402d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a6215971eda4de2bbb2fbf88c75d70f in term 0.
I20260808 11:38:29.225195 5148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31ffcaf64da44826a308a5f7de96a409" candidate_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" is_pre_election: true
I20260808 11:38:29.225466 5148 raft_consensus.cc:2454] T 31ffcaf64da44826a308a5f7de96a409 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a6215971eda4de2bbb2fbf88c75d70f in term 0.
I20260808 11:38:29.226009 4603 leader_election.cc:302] T 31ffcaf64da44826a308a5f7de96a409 P 5a6215971eda4de2bbb2fbf88c75d70f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a6215971eda4de2bbb2fbf88c75d70f, ab0c3e1d88d14a1290d8abdf88f84d10; no voters:
I20260808 11:38:29.226329 5341 raft_consensus.cc:2781] T 31ffcaf64da44826a308a5f7de96a409 P 5a6215971eda4de2bbb2fbf88c75d70f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:38:29.226454 5341 raft_consensus.cc:482] T 31ffcaf64da44826a308a5f7de96a409 P 5a6215971eda4de2bbb2fbf88c75d70f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:38:29.226553 5341 raft_consensus.cc:3037] T 31ffcaf64da44826a308a5f7de96a409 P 5a6215971eda4de2bbb2fbf88c75d70f [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:29.229799 5341 raft_consensus.cc:504] T 31ffcaf64da44826a308a5f7de96a409 P 5a6215971eda4de2bbb2fbf88c75d70f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } }
I20260808 11:38:29.230479 5341 leader_election.cc:288] T 31ffcaf64da44826a308a5f7de96a409 P 5a6215971eda4de2bbb2fbf88c75d70f [CANDIDATE]: Term 1 election: Requested vote from peers b6f5a3d8e29947948e3f0ed47f43402d (127.24.76.66:42771), ab0c3e1d88d14a1290d8abdf88f84d10 (127.24.76.67:41839)
I20260808 11:38:29.231065 4936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31ffcaf64da44826a308a5f7de96a409" candidate_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6f5a3d8e29947948e3f0ed47f43402d"
I20260808 11:38:29.231276 4936 raft_consensus.cc:3037] T 31ffcaf64da44826a308a5f7de96a409 P b6f5a3d8e29947948e3f0ed47f43402d [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:29.231259 5148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31ffcaf64da44826a308a5f7de96a409" candidate_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10"
I20260808 11:38:29.231467 5148 raft_consensus.cc:3037] T 31ffcaf64da44826a308a5f7de96a409 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:29.234529 4936 raft_consensus.cc:2454] T 31ffcaf64da44826a308a5f7de96a409 P b6f5a3d8e29947948e3f0ed47f43402d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a6215971eda4de2bbb2fbf88c75d70f in term 1.
I20260808 11:38:29.234542 5148 raft_consensus.cc:2454] T 31ffcaf64da44826a308a5f7de96a409 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a6215971eda4de2bbb2fbf88c75d70f in term 1.
I20260808 11:38:29.234951 4603 leader_election.cc:302] T 31ffcaf64da44826a308a5f7de96a409 P 5a6215971eda4de2bbb2fbf88c75d70f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a6215971eda4de2bbb2fbf88c75d70f, b6f5a3d8e29947948e3f0ed47f43402d; no voters:
I20260808 11:38:29.235556 5341 raft_consensus.cc:2781] T 31ffcaf64da44826a308a5f7de96a409 P 5a6215971eda4de2bbb2fbf88c75d70f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:38:29.235704 5341 raft_consensus.cc:686] T 31ffcaf64da44826a308a5f7de96a409 P 5a6215971eda4de2bbb2fbf88c75d70f [term 1 LEADER]: Becoming Leader. State: Replica: 5a6215971eda4de2bbb2fbf88c75d70f, State: Running, Role: LEADER
I20260808 11:38:29.235977 5341 consensus_queue.cc:237] T 31ffcaf64da44826a308a5f7de96a409 P 5a6215971eda4de2bbb2fbf88c75d70f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } }
I20260808 11:38:29.238809 4462 catalog_manager.cc:5495] T 31ffcaf64da44826a308a5f7de96a409 P 5a6215971eda4de2bbb2fbf88c75d70f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a6215971eda4de2bbb2fbf88c75d70f (127.24.76.65). New cstate: current_term: 1 leader_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } health_report { overall_health: UNKNOWN } } }
I20260808 11:38:29.268790 5333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:29.262465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60270 (local address 127.24.76.65:43047)
0808 11:38:29.262673 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:29.262679 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:29.262700 (+ 21us) server_negotiation.cc:407] Connection header received
0808 11:38:29.262787 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:29.262794 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:29.262887 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:29.262987 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:38:29.263348 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:29.264510 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:29.266533 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:29.267183 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:29.267305 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:29.267966 (+ 661us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:38:29.268120 (+ 154us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:38:29.268210 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:29.268396 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 11:38:29.268475 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 11:38:29.272140 5148 raft_consensus.cc:1261] T 31ffcaf64da44826a308a5f7de96a409 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 1 FOLLOWER]: Refusing update from remote peer 5a6215971eda4de2bbb2fbf88c75d70f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:38:29.272128 4936 raft_consensus.cc:1261] T 31ffcaf64da44826a308a5f7de96a409 P b6f5a3d8e29947948e3f0ed47f43402d [term 1 FOLLOWER]: Refusing update from remote peer 5a6215971eda4de2bbb2fbf88c75d70f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:38:29.272800 5274 consensus_queue.cc:1035] T 31ffcaf64da44826a308a5f7de96a409 P 5a6215971eda4de2bbb2fbf88c75d70f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:38:29.273069 5341 consensus_queue.cc:1035] T 31ffcaf64da44826a308a5f7de96a409 P 5a6215971eda4de2bbb2fbf88c75d70f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:38:29.298866 4462 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:38:29.299131 4462 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:38:29.305006 4462 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 508408a84d5c4b838a2198a913aff5a2: Sending DeleteTablet for 3 replicas of tablet 31ffcaf64da44826a308a5f7de96a409
I20260808 11:38:29.306080 4906 tablet_service.cc:1514] Processing DeleteTablet for tablet 31ffcaf64da44826a308a5f7de96a409 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:38:29 UTC) from {username='oryx', principal='oryx/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:43306
I20260808 11:38:29.306391 5300 tablet_replica.cc:330] T 31ffcaf64da44826a308a5f7de96a409 P b6f5a3d8e29947948e3f0ed47f43402d: stopping tablet replica
I20260808 11:38:29.306589 5300 raft_consensus.cc:2229] T 31ffcaf64da44826a308a5f7de96a409 P b6f5a3d8e29947948e3f0ed47f43402d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:38:29.306835 5300 raft_consensus.cc:2258] T 31ffcaf64da44826a308a5f7de96a409 P b6f5a3d8e29947948e3f0ed47f43402d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:38:29.307598 4906 tablet_service.cc:1514] Processing DeleteTablet for tablet 31ffcaf64da44826a308a5f7de96a409 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:38:29 UTC) from {username='oryx', principal='oryx/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:43306
I20260808 11:38:29.307777 4673 tablet_service.cc:1514] Processing DeleteTablet for tablet 31ffcaf64da44826a308a5f7de96a409 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:38:29 UTC) from {username='oryx', principal='oryx/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:60252
I20260808 11:38:29.308104 5349 tablet_replica.cc:330] T 31ffcaf64da44826a308a5f7de96a409 P 5a6215971eda4de2bbb2fbf88c75d70f: stopping tablet replica
I20260808 11:38:29.308328 5349 raft_consensus.cc:2229] T 31ffcaf64da44826a308a5f7de96a409 P 5a6215971eda4de2bbb2fbf88c75d70f [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:38:29.308570 5107 tablet_service.cc:1514] Processing DeleteTablet for tablet 31ffcaf64da44826a308a5f7de96a409 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:38:29 UTC) from {username='oryx', principal='oryx/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:33770
I20260808 11:38:29.308657 5349 raft_consensus.cc:2258] T 31ffcaf64da44826a308a5f7de96a409 P 5a6215971eda4de2bbb2fbf88c75d70f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:38:29.308899 5299 tablet_replica.cc:330] T 31ffcaf64da44826a308a5f7de96a409 P ab0c3e1d88d14a1290d8abdf88f84d10: stopping tablet replica
I20260808 11:38:29.309065 5299 raft_consensus.cc:2229] T 31ffcaf64da44826a308a5f7de96a409 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:38:29.309298 5299 raft_consensus.cc:2258] T 31ffcaf64da44826a308a5f7de96a409 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:38:29.310140 5349 ts_tablet_manager.cc:1905] T 31ffcaf64da44826a308a5f7de96a409 P 5a6215971eda4de2bbb2fbf88c75d70f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:38:29.311241 4461 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46378:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:38:29.311972 4461 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:38:29.315234 5300 ts_tablet_manager.cc:1905] T 31ffcaf64da44826a308a5f7de96a409 P b6f5a3d8e29947948e3f0ed47f43402d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:38:29.318845 5299 ts_tablet_manager.cc:1905] T 31ffcaf64da44826a308a5f7de96a409 P ab0c3e1d88d14a1290d8abdf88f84d10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:38:29.319193 5349 ts_tablet_manager.cc:1918] T 31ffcaf64da44826a308a5f7de96a409 P 5a6215971eda4de2bbb2fbf88c75d70f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:38:29.319413 5349 log.cc:1196] T 31ffcaf64da44826a308a5f7de96a409 P 5a6215971eda4de2bbb2fbf88c75d70f: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-0/wal/wals/31ffcaf64da44826a308a5f7de96a409
I20260808 11:38:29.319897 5349 ts_tablet_manager.cc:1939] T 31ffcaf64da44826a308a5f7de96a409 P 5a6215971eda4de2bbb2fbf88c75d70f: Deleting consensus metadata
I20260808 11:38:29.322263 4441 catalog_manager.cc:4841] TS 5a6215971eda4de2bbb2fbf88c75d70f (127.24.76.65:43047): tablet 31ffcaf64da44826a308a5f7de96a409 (table test-table [id=f7dcfa1e85034a1c8c630fb993e7e2f1]) successfully deleted
I20260808 11:38:29.326273 4673 tablet_service.cc:1467] Processing CreateTablet for tablet 172cd03271fc4ecf8f15ddecc0f3fd57 (DEFAULT_TABLE table=test-table [id=d1b7ad1036bd4e2f9a8693385465d402]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:38:29.326731 4673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 172cd03271fc4ecf8f15ddecc0f3fd57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:29.330803 4906 tablet_service.cc:1467] Processing CreateTablet for tablet 172cd03271fc4ecf8f15ddecc0f3fd57 (DEFAULT_TABLE table=test-table [id=d1b7ad1036bd4e2f9a8693385465d402]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:38:29.330895 5107 tablet_service.cc:1467] Processing CreateTablet for tablet 172cd03271fc4ecf8f15ddecc0f3fd57 (DEFAULT_TABLE table=test-table [id=d1b7ad1036bd4e2f9a8693385465d402]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:38:29.331187 4906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 172cd03271fc4ecf8f15ddecc0f3fd57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:29.331314 5107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 172cd03271fc4ecf8f15ddecc0f3fd57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:29.340785 5259 tablet_bootstrap.cc:492] T 172cd03271fc4ecf8f15ddecc0f3fd57 P b6f5a3d8e29947948e3f0ed47f43402d: Bootstrap starting.
I20260808 11:38:29.342604 5259 tablet_bootstrap.cc:654] T 172cd03271fc4ecf8f15ddecc0f3fd57 P b6f5a3d8e29947948e3f0ed47f43402d: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:29.344301 5258 tablet_bootstrap.cc:492] T 172cd03271fc4ecf8f15ddecc0f3fd57 P ab0c3e1d88d14a1290d8abdf88f84d10: Bootstrap starting.
I20260808 11:38:29.345952 5257 tablet_bootstrap.cc:492] T 172cd03271fc4ecf8f15ddecc0f3fd57 P 5a6215971eda4de2bbb2fbf88c75d70f: Bootstrap starting.
I20260808 11:38:29.346407 5258 tablet_bootstrap.cc:654] T 172cd03271fc4ecf8f15ddecc0f3fd57 P ab0c3e1d88d14a1290d8abdf88f84d10: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:29.347690 5257 tablet_bootstrap.cc:654] T 172cd03271fc4ecf8f15ddecc0f3fd57 P 5a6215971eda4de2bbb2fbf88c75d70f: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:29.352613 5299 ts_tablet_manager.cc:1918] T 31ffcaf64da44826a308a5f7de96a409 P ab0c3e1d88d14a1290d8abdf88f84d10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:38:29.352772 5299 log.cc:1196] T 31ffcaf64da44826a308a5f7de96a409 P ab0c3e1d88d14a1290d8abdf88f84d10: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-2/wal/wals/31ffcaf64da44826a308a5f7de96a409
I20260808 11:38:29.353226 5299 ts_tablet_manager.cc:1939] T 31ffcaf64da44826a308a5f7de96a409 P ab0c3e1d88d14a1290d8abdf88f84d10: Deleting consensus metadata
I20260808 11:38:29.354758 5300 ts_tablet_manager.cc:1918] T 31ffcaf64da44826a308a5f7de96a409 P b6f5a3d8e29947948e3f0ed47f43402d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:38:29.354924 5300 log.cc:1196] T 31ffcaf64da44826a308a5f7de96a409 P b6f5a3d8e29947948e3f0ed47f43402d: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-1/wal/wals/31ffcaf64da44826a308a5f7de96a409
I20260808 11:38:29.355348 5300 ts_tablet_manager.cc:1939] T 31ffcaf64da44826a308a5f7de96a409 P b6f5a3d8e29947948e3f0ed47f43402d: Deleting consensus metadata
I20260808 11:38:29.358616 4440 catalog_manager.cc:4841] TS ab0c3e1d88d14a1290d8abdf88f84d10 (127.24.76.67:41839): tablet 31ffcaf64da44826a308a5f7de96a409 (table test-table [id=f7dcfa1e85034a1c8c630fb993e7e2f1]) successfully deleted
I20260808 11:38:29.360172 4440 catalog_manager.cc:4841] TS b6f5a3d8e29947948e3f0ed47f43402d (127.24.76.66:42771): tablet 31ffcaf64da44826a308a5f7de96a409 (table test-table [id=f7dcfa1e85034a1c8c630fb993e7e2f1]) successfully deleted
W20260808 11:38:29.361706 4440 catalog_manager.cc:4808] TS b6f5a3d8e29947948e3f0ed47f43402d (127.24.76.66:42771): delete failed for tablet 31ffcaf64da44826a308a5f7de96a409 because the tablet was not found. No further retry: Not found: Tablet not found: 31ffcaf64da44826a308a5f7de96a409
I20260808 11:38:29.365726 5257 tablet_bootstrap.cc:492] T 172cd03271fc4ecf8f15ddecc0f3fd57 P 5a6215971eda4de2bbb2fbf88c75d70f: No bootstrap required, opened a new log
I20260808 11:38:29.365734 5258 tablet_bootstrap.cc:492] T 172cd03271fc4ecf8f15ddecc0f3fd57 P ab0c3e1d88d14a1290d8abdf88f84d10: No bootstrap required, opened a new log
I20260808 11:38:29.365945 5258 ts_tablet_manager.cc:1397] T 172cd03271fc4ecf8f15ddecc0f3fd57 P ab0c3e1d88d14a1290d8abdf88f84d10: Time spent bootstrapping tablet: real 0.022s user 0.004s sys 0.004s
I20260808 11:38:29.366050 5257 ts_tablet_manager.cc:1397] T 172cd03271fc4ecf8f15ddecc0f3fd57 P 5a6215971eda4de2bbb2fbf88c75d70f: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.004s
I20260808 11:38:29.366768 5258 raft_consensus.cc:348] T 172cd03271fc4ecf8f15ddecc0f3fd57 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } }
I20260808 11:38:29.367058 5258 raft_consensus.cc:374] T 172cd03271fc4ecf8f15ddecc0f3fd57 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:29.367147 5258 raft_consensus.cc:729] T 172cd03271fc4ecf8f15ddecc0f3fd57 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab0c3e1d88d14a1290d8abdf88f84d10, State: Initialized, Role: FOLLOWER
I20260808 11:38:29.366911 5257 raft_consensus.cc:348] T 172cd03271fc4ecf8f15ddecc0f3fd57 P 5a6215971eda4de2bbb2fbf88c75d70f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } }
I20260808 11:38:29.367380 5257 raft_consensus.cc:374] T 172cd03271fc4ecf8f15ddecc0f3fd57 P 5a6215971eda4de2bbb2fbf88c75d70f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:29.367494 5257 raft_consensus.cc:729] T 172cd03271fc4ecf8f15ddecc0f3fd57 P 5a6215971eda4de2bbb2fbf88c75d70f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a6215971eda4de2bbb2fbf88c75d70f, State: Initialized, Role: FOLLOWER
I20260808 11:38:29.367436 5258 consensus_queue.cc:260] T 172cd03271fc4ecf8f15ddecc0f3fd57 P ab0c3e1d88d14a1290d8abdf88f84d10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } }
I20260808 11:38:29.367854 5257 consensus_queue.cc:260] T 172cd03271fc4ecf8f15ddecc0f3fd57 P 5a6215971eda4de2bbb2fbf88c75d70f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } }
I20260808 11:38:29.371088 5258 ts_tablet_manager.cc:1428] T 172cd03271fc4ecf8f15ddecc0f3fd57 P ab0c3e1d88d14a1290d8abdf88f84d10: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260808 11:38:29.371629 5257 ts_tablet_manager.cc:1428] T 172cd03271fc4ecf8f15ddecc0f3fd57 P 5a6215971eda4de2bbb2fbf88c75d70f: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260808 11:38:29.375059 5259 tablet_bootstrap.cc:492] T 172cd03271fc4ecf8f15ddecc0f3fd57 P b6f5a3d8e29947948e3f0ed47f43402d: No bootstrap required, opened a new log
I20260808 11:38:29.375247 5259 ts_tablet_manager.cc:1397] T 172cd03271fc4ecf8f15ddecc0f3fd57 P b6f5a3d8e29947948e3f0ed47f43402d: Time spent bootstrapping tablet: real 0.035s user 0.001s sys 0.011s
I20260808 11:38:29.376070 5259 raft_consensus.cc:348] T 172cd03271fc4ecf8f15ddecc0f3fd57 P b6f5a3d8e29947948e3f0ed47f43402d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } }
I20260808 11:38:29.376370 5259 raft_consensus.cc:374] T 172cd03271fc4ecf8f15ddecc0f3fd57 P b6f5a3d8e29947948e3f0ed47f43402d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:29.376466 5259 raft_consensus.cc:729] T 172cd03271fc4ecf8f15ddecc0f3fd57 P b6f5a3d8e29947948e3f0ed47f43402d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6f5a3d8e29947948e3f0ed47f43402d, State: Initialized, Role: FOLLOWER
I20260808 11:38:29.376765 5259 consensus_queue.cc:260] T 172cd03271fc4ecf8f15ddecc0f3fd57 P b6f5a3d8e29947948e3f0ed47f43402d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } }
I20260808 11:38:29.377303 5259 ts_tablet_manager.cc:1428] T 172cd03271fc4ecf8f15ddecc0f3fd57 P b6f5a3d8e29947948e3f0ed47f43402d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 11:38:29.403039 5271 raft_consensus.cc:482] T 172cd03271fc4ecf8f15ddecc0f3fd57 P b6f5a3d8e29947948e3f0ed47f43402d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:38:29.403264 5271 raft_consensus.cc:504] T 172cd03271fc4ecf8f15ddecc0f3fd57 P b6f5a3d8e29947948e3f0ed47f43402d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } }
I20260808 11:38:29.403926 5271 leader_election.cc:288] T 172cd03271fc4ecf8f15ddecc0f3fd57 P b6f5a3d8e29947948e3f0ed47f43402d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a6215971eda4de2bbb2fbf88c75d70f (127.24.76.65:43047), ab0c3e1d88d14a1290d8abdf88f84d10 (127.24.76.67:41839)
I20260808 11:38:29.404541 4702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "172cd03271fc4ecf8f15ddecc0f3fd57" candidate_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" is_pre_election: true
I20260808 11:38:29.404627 5147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "172cd03271fc4ecf8f15ddecc0f3fd57" candidate_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" is_pre_election: true
I20260808 11:38:29.404817 4702 raft_consensus.cc:2454] T 172cd03271fc4ecf8f15ddecc0f3fd57 P 5a6215971eda4de2bbb2fbf88c75d70f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6f5a3d8e29947948e3f0ed47f43402d in term 0.
I20260808 11:38:29.404855 5147 raft_consensus.cc:2454] T 172cd03271fc4ecf8f15ddecc0f3fd57 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6f5a3d8e29947948e3f0ed47f43402d in term 0.
I20260808 11:38:29.405336 4827 leader_election.cc:302] T 172cd03271fc4ecf8f15ddecc0f3fd57 P b6f5a3d8e29947948e3f0ed47f43402d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5a6215971eda4de2bbb2fbf88c75d70f, ab0c3e1d88d14a1290d8abdf88f84d10, b6f5a3d8e29947948e3f0ed47f43402d; no voters:
I20260808 11:38:29.405683 5271 raft_consensus.cc:2781] T 172cd03271fc4ecf8f15ddecc0f3fd57 P b6f5a3d8e29947948e3f0ed47f43402d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:38:29.405799 5271 raft_consensus.cc:482] T 172cd03271fc4ecf8f15ddecc0f3fd57 P b6f5a3d8e29947948e3f0ed47f43402d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:38:29.405885 5271 raft_consensus.cc:3037] T 172cd03271fc4ecf8f15ddecc0f3fd57 P b6f5a3d8e29947948e3f0ed47f43402d [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:29.408779 5271 raft_consensus.cc:504] T 172cd03271fc4ecf8f15ddecc0f3fd57 P b6f5a3d8e29947948e3f0ed47f43402d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } }
I20260808 11:38:29.410039 5147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "172cd03271fc4ecf8f15ddecc0f3fd57" candidate_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10"
I20260808 11:38:29.410243 5147 raft_consensus.cc:3037] T 172cd03271fc4ecf8f15ddecc0f3fd57 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:29.409971 4702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "172cd03271fc4ecf8f15ddecc0f3fd57" candidate_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a6215971eda4de2bbb2fbf88c75d70f"
I20260808 11:38:29.410539 4702 raft_consensus.cc:3037] T 172cd03271fc4ecf8f15ddecc0f3fd57 P 5a6215971eda4de2bbb2fbf88c75d70f [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:29.413120 5147 raft_consensus.cc:2454] T 172cd03271fc4ecf8f15ddecc0f3fd57 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6f5a3d8e29947948e3f0ed47f43402d in term 1.
I20260808 11:38:29.413822 4827 leader_election.cc:302] T 172cd03271fc4ecf8f15ddecc0f3fd57 P b6f5a3d8e29947948e3f0ed47f43402d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab0c3e1d88d14a1290d8abdf88f84d10, b6f5a3d8e29947948e3f0ed47f43402d; no voters:
I20260808 11:38:29.414142 5278 raft_consensus.cc:2781] T 172cd03271fc4ecf8f15ddecc0f3fd57 P b6f5a3d8e29947948e3f0ed47f43402d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:38:29.414122 4702 raft_consensus.cc:2454] T 172cd03271fc4ecf8f15ddecc0f3fd57 P 5a6215971eda4de2bbb2fbf88c75d70f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6f5a3d8e29947948e3f0ed47f43402d in term 1.
I20260808 11:38:29.414274 5278 raft_consensus.cc:686] T 172cd03271fc4ecf8f15ddecc0f3fd57 P b6f5a3d8e29947948e3f0ed47f43402d [term 1 LEADER]: Becoming Leader. State: Replica: b6f5a3d8e29947948e3f0ed47f43402d, State: Running, Role: LEADER
I20260808 11:38:29.414562 5278 consensus_queue.cc:237] T 172cd03271fc4ecf8f15ddecc0f3fd57 P b6f5a3d8e29947948e3f0ed47f43402d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } }
I20260808 11:38:29.416291 5271 leader_election.cc:288] T 172cd03271fc4ecf8f15ddecc0f3fd57 P b6f5a3d8e29947948e3f0ed47f43402d [CANDIDATE]: Term 1 election: Requested vote from peers 5a6215971eda4de2bbb2fbf88c75d70f (127.24.76.65:43047), ab0c3e1d88d14a1290d8abdf88f84d10 (127.24.76.67:41839)
I20260808 11:38:29.417272 4461 catalog_manager.cc:5495] T 172cd03271fc4ecf8f15ddecc0f3fd57 P b6f5a3d8e29947948e3f0ed47f43402d reported cstate change: term changed from 0 to 1, leader changed from <none> to b6f5a3d8e29947948e3f0ed47f43402d (127.24.76.66). New cstate: current_term: 1 leader_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6f5a3d8e29947948e3f0ed47f43402d" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 } health_report { overall_health: UNKNOWN } } }
I20260808 11:38:29.440279 5380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:29.434480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43320 (local address 127.24.76.66:42771)
0808 11:38:29.435405 (+ 925us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:29.435410 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:29.435424 (+ 14us) server_negotiation.cc:407] Connection header received
0808 11:38:29.435472 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:29.435475 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:29.435531 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:29.435614 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:38:29.436019 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:29.436708 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:29.438181 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:29.438845 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:29.438984 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:29.439490 (+ 506us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:38:29.439636 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:38:29.439734 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:29.439878 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 11:38:29.439962 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":822,"thread_start_us":710,"threads_started":1}
I20260808 11:38:29.455111 5380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:29.449187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:33823 (local address 127.24.76.66:42771)
0808 11:38:29.449520 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:29.449526 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:29.449546 (+ 20us) server_negotiation.cc:407] Connection header received
0808 11:38:29.449601 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:29.449606 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:29.449684 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:29.449778 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:38:29.450206 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:29.451278 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:29.453154 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:29.453795 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:29.453948 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:29.454390 (+ 442us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:38:29.454521 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:38:29.454611 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:29.454730 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 11:38:29.454829 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":224}
I20260808 11:38:29.455404 5346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:29.449216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:42771 (local address 127.24.76.66:33823)
0808 11:38:29.449389 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:29.449406 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:29.449491 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:29.449807 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:29.449812 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:29.449825 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 11:38:29.450053 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:29.450063 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:29.451421 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:29.451427 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:29.452997 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:29.453011 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:29.454000 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:29.454007 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:29.454181 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:29.454227 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 11:38:29.454620 (+ 393us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 11:38:29.454630 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 11:38:29.454703 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 11:38:29.454896 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":35}
I20260808 11:38:29.464148 5147 raft_consensus.cc:1261] T 172cd03271fc4ecf8f15ddecc0f3fd57 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 1 FOLLOWER]: Refusing update from remote peer b6f5a3d8e29947948e3f0ed47f43402d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:38:29.464329 4702 raft_consensus.cc:1261] T 172cd03271fc4ecf8f15ddecc0f3fd57 P 5a6215971eda4de2bbb2fbf88c75d70f [term 1 FOLLOWER]: Refusing update from remote peer b6f5a3d8e29947948e3f0ed47f43402d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:38:29.464871 5278 consensus_queue.cc:1035] T 172cd03271fc4ecf8f15ddecc0f3fd57 P b6f5a3d8e29947948e3f0ed47f43402d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab0c3e1d88d14a1290d8abdf88f84d10" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:38:29.465170 5271 consensus_queue.cc:1035] T 172cd03271fc4ecf8f15ddecc0f3fd57 P b6f5a3d8e29947948e3f0ed47f43402d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a6215971eda4de2bbb2fbf88c75d70f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:38:29.517292 4461 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:38:29.517594 4461 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:38:29.520637 4461 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 508408a84d5c4b838a2198a913aff5a2: Sending DeleteTablet for 3 replicas of tablet 172cd03271fc4ecf8f15ddecc0f3fd57
I20260808 11:38:29.521337 4673 tablet_service.cc:1514] Processing DeleteTablet for tablet 172cd03271fc4ecf8f15ddecc0f3fd57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:38:29 UTC) from {username='oryx', principal='oryx/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:60252
I20260808 11:38:29.521564 5107 tablet_service.cc:1514] Processing DeleteTablet for tablet 172cd03271fc4ecf8f15ddecc0f3fd57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:38:29 UTC) from {username='oryx', principal='oryx/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:33770
I20260808 11:38:29.521642 5349 tablet_replica.cc:330] T 172cd03271fc4ecf8f15ddecc0f3fd57 P 5a6215971eda4de2bbb2fbf88c75d70f: stopping tablet replica
I20260808 11:38:29.521670 4906 tablet_service.cc:1514] Processing DeleteTablet for tablet 172cd03271fc4ecf8f15ddecc0f3fd57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:38:29 UTC) from {username='oryx', principal='oryx/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:43306
I20260808 11:38:29.521824 5299 tablet_replica.cc:330] T 172cd03271fc4ecf8f15ddecc0f3fd57 P ab0c3e1d88d14a1290d8abdf88f84d10: stopping tablet replica
I20260808 11:38:29.521819 5349 raft_consensus.cc:2229] T 172cd03271fc4ecf8f15ddecc0f3fd57 P 5a6215971eda4de2bbb2fbf88c75d70f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:38:29.521958 5300 tablet_replica.cc:330] T 172cd03271fc4ecf8f15ddecc0f3fd57 P b6f5a3d8e29947948e3f0ed47f43402d: stopping tablet replica
I20260808 11:38:29.522055 5299 raft_consensus.cc:2229] T 172cd03271fc4ecf8f15ddecc0f3fd57 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:38:29.522153 5300 raft_consensus.cc:2229] T 172cd03271fc4ecf8f15ddecc0f3fd57 P b6f5a3d8e29947948e3f0ed47f43402d [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:38:29.522233 5299 raft_consensus.cc:2258] T 172cd03271fc4ecf8f15ddecc0f3fd57 P ab0c3e1d88d14a1290d8abdf88f84d10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:38:29.522294 5349 raft_consensus.cc:2258] T 172cd03271fc4ecf8f15ddecc0f3fd57 P 5a6215971eda4de2bbb2fbf88c75d70f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:38:29.522449 5300 raft_consensus.cc:2258] T 172cd03271fc4ecf8f15ddecc0f3fd57 P b6f5a3d8e29947948e3f0ed47f43402d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:38:29.523315 5299 ts_tablet_manager.cc:1905] T 172cd03271fc4ecf8f15ddecc0f3fd57 P ab0c3e1d88d14a1290d8abdf88f84d10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:38:29.523535 5300 ts_tablet_manager.cc:1905] T 172cd03271fc4ecf8f15ddecc0f3fd57 P b6f5a3d8e29947948e3f0ed47f43402d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:38:29.523531 5349 ts_tablet_manager.cc:1905] T 172cd03271fc4ecf8f15ddecc0f3fd57 P 5a6215971eda4de2bbb2fbf88c75d70f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:38:29.529815 5299 ts_tablet_manager.cc:1918] T 172cd03271fc4ecf8f15ddecc0f3fd57 P ab0c3e1d88d14a1290d8abdf88f84d10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 11:38:29.529947 5300 ts_tablet_manager.cc:1918] T 172cd03271fc4ecf8f15ddecc0f3fd57 P b6f5a3d8e29947948e3f0ed47f43402d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 11:38:29.530093 5300 log.cc:1196] T 172cd03271fc4ecf8f15ddecc0f3fd57 P b6f5a3d8e29947948e3f0ed47f43402d: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-1/wal/wals/172cd03271fc4ecf8f15ddecc0f3fd57
I20260808 11:38:29.530196 5299 log.cc:1196] T 172cd03271fc4ecf8f15ddecc0f3fd57 P ab0c3e1d88d14a1290d8abdf88f84d10: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-2/wal/wals/172cd03271fc4ecf8f15ddecc0f3fd57
I20260808 11:38:29.530489 5300 ts_tablet_manager.cc:1939] T 172cd03271fc4ecf8f15ddecc0f3fd57 P b6f5a3d8e29947948e3f0ed47f43402d: Deleting consensus metadata
I20260808 11:38:29.530591 5299 ts_tablet_manager.cc:1939] T 172cd03271fc4ecf8f15ddecc0f3fd57 P ab0c3e1d88d14a1290d8abdf88f84d10: Deleting consensus metadata
I20260808 11:38:29.531565 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 4575
I20260808 11:38:29.531545 4440 catalog_manager.cc:4841] TS ab0c3e1d88d14a1290d8abdf88f84d10 (127.24.76.67:41839): tablet 172cd03271fc4ecf8f15ddecc0f3fd57 (table test-table [id=d1b7ad1036bd4e2f9a8693385465d402]) successfully deleted
I20260808 11:38:29.531769 4440 catalog_manager.cc:4841] TS b6f5a3d8e29947948e3f0ed47f43402d (127.24.76.66:42771): tablet 172cd03271fc4ecf8f15ddecc0f3fd57 (table test-table [id=d1b7ad1036bd4e2f9a8693385465d402]) successfully deleted
I20260808 11:38:29.533963 5349 ts_tablet_manager.cc:1918] T 172cd03271fc4ecf8f15ddecc0f3fd57 P 5a6215971eda4de2bbb2fbf88c75d70f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 11:38:29.534116 5349 log.cc:1196] T 172cd03271fc4ecf8f15ddecc0f3fd57 P 5a6215971eda4de2bbb2fbf88c75d70f: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786189042529715-24881-0/minicluster-data/ts-0/wal/wals/172cd03271fc4ecf8f15ddecc0f3fd57
I20260808 11:38:29.534611 5349 ts_tablet_manager.cc:1939] T 172cd03271fc4ecf8f15ddecc0f3fd57 P 5a6215971eda4de2bbb2fbf88c75d70f: Deleting consensus metadata
I20260808 11:38:29.536876 4441 catalog_manager.cc:4841] TS 5a6215971eda4de2bbb2fbf88c75d70f (127.24.76.65:43047): tablet 172cd03271fc4ecf8f15ddecc0f3fd57 (table test-table [id=d1b7ad1036bd4e2f9a8693385465d402]) successfully deleted
Aug 08 11:38:29 dist-test-slave-tc2s krb5kdc[4388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.76.65@KRBTEST.COM
I20260808 11:38:29.549549 5333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:29.532790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60280 (local address 127.24.76.65:43047)
0808 11:38:29.532963 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:29.532969 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:29.534489 (+ 1520us) server_negotiation.cc:407] Connection header received
0808 11:38:29.534750 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:29.534758 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:29.534868 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:29.534966 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:29.536569 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:29.537695 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:29.538956 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:29.539574 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:29.539686 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:29.544006 (+ 4320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:29.544020 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:29.544037 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:29.544069 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:29.547023 (+ 2954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:29.547746 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:29.547753 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:29.547755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:29.547895 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:29.548250 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:29.548253 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:29.548255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:29.548587 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:29.548826 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:29.549115 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 11:38:29.549271 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 11:38:29.550244 4775 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:29.675846 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 4575
I20260808 11:38:29.712706 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 4796
Aug 08 11:38:29 dist-test-slave-tc2s krb5kdc[4388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.76.66@KRBTEST.COM
I20260808 11:38:29.726845 5380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:29.713510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43322 (local address 127.24.76.66:42771)
0808 11:38:29.713717 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:29.713723 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:29.713743 (+ 20us) server_negotiation.cc:407] Connection header received
0808 11:38:29.713807 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:29.713811 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:29.713870 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:29.713988 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:29.715201 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:29.715946 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:29.717240 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:29.717668 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:29.717774 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:29.721779 (+ 4005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:29.721792 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:29.721807 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:29.721844 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:29.724580 (+ 2736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:29.725140 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:29.725147 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:29.725149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:29.725287 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:29.725576 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:29.725579 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:29.725581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:29.725973 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:29.726211 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:29.726434 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 11:38:29.726590 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 11:38:29.727438 5002 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:29.875113 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 4796
I20260808 11:38:29.906041 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 5015
Aug 08 11:38:29 dist-test-slave-tc2s krb5kdc[4388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.76.67@KRBTEST.COM
I20260808 11:38:29.920137 5400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:29.906778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33780 (local address 127.24.76.67:41839)
0808 11:38:29.907212 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:29.907216 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:29.907233 (+ 17us) server_negotiation.cc:407] Connection header received
0808 11:38:29.907290 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:29.907294 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:29.907354 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:29.907465 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:29.908725 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:29.909433 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:29.910708 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:29.911101 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:29.911198 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:29.915238 (+ 4040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:29.915254 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:29.915271 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:29.915303 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:29.917953 (+ 2650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:29.918558 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:29.918564 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:29.918566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:29.918682 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:29.918944 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:29.918947 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:29.918949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:29.919283 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:29.919519 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:29.919713 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 11:38:29.919872 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":222,"threads_started":1}
I20260808 11:38:29.920742 5213 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:30.014856 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 5015
I20260808 11:38:30.037850 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 4409
I20260808 11:38:30.039031 4515 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:30.155423 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 4409
2026-08-08T11:38:30Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6586 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:38:30 dist-test-slave-tc2s krb5kdc[5413](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:38:30 dist-test-slave-tc2s krb5kdc[5413](info): set up 2 sockets
Aug 08 11:38:30 dist-test-slave-tc2s krb5kdc[5413](info): commencing operation
krb5kdc: starting...
W20260808 11:38:31.369719 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:38:31 dist-test-slave-tc2s krb5kdc[5413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189111, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:38:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:38:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
I20260808 11:38:31.605865 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:39711
--webserver_interface=127.24.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:41333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:39711
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:38:31.924490 5430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:31.924747 5430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:31.924851 5430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:31.931814 5430 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:38:31.931912 5430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:31.931964 5430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:31.932019 5430 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:38:31.932065 5430 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:38:31.941725 5430 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:41333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:39711
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:39711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:31.942884 5430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:31.944087 5430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:31.955163 5444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:31.955116 5440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:31.955201 5441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:31.956517 5430 server_base.cc:1034] running on GCE node
I20260808 11:38:31.957209 5430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:31.958559 5430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:31.959749 5430 hybrid_clock.cc:648] HybridClock initialized: now 1786189111959713 us; error 26 us; skew 500 ppm
I20260808 11:38:31.960057 5430 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T11:38:32Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1870 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:38:32 dist-test-slave-tc2s krb5kdc[5451](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:38:32 dist-test-slave-tc2s krb5kdc[5451](info): set up 2 sockets
Aug 08 11:38:32 dist-test-slave-tc2s krb5kdc[5451](info): commencing operation
krb5kdc: starting...
W20260808 11:38:33.234202 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.001s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:38:33 dist-test-slave-tc2s krb5kdc[5451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:38:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:38:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
I20260808 11:38:33.475184 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:44953
--webserver_interface=127.24.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:34965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:44953
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:38:33.795631 5468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:33.795899 5468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:33.796028 5468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:33.803020 5468 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:38:33.803117 5468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:33.803194 5468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:33.803270 5468 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:38:33.803329 5468 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:38:33.812819 5468 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:34965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:44953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:44953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:33.814087 5468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:33.815342 5468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:33.826583 5480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:33.826548 5483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:33.826767 5478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:33.827646 5468 server_base.cc:1034] running on GCE node
I20260808 11:38:33.828351 5468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:33.829663 5468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:33.830848 5468 hybrid_clock.cc:648] HybridClock initialized: now 1786189113830785 us; error 55 us; skew 500 ppm
I20260808 11:38:33.831180 5468 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:38:33 dist-test-slave-tc2s krb5kdc[5451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189113, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:33.834167 5468 init.cc:377] Logged in from keytab as kudu/127.24.76.126@KRBTEST.COM (short username kudu)
I20260808 11:38:33.835613 5468 webserver.cc:460] Webserver started at http://127.24.76.126:38821/ using document root <none> and password file <none>
I20260808 11:38:33.836273 5468 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:33.836391 5468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:33.836691 5468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:33.839684 5468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "1c51f9ff61e94208855cfb2586b890f8"
format_stamp: "Formatted at 2026-08-08 11:38:33 on dist-test-slave-tc2s"
server_key: "68d0d655c682fe3d73d8e2d8240ecb95"
server_key_iv: "b26d0490eaea548bef67eac8f4c340dc"
server_key_version: "encryptionkey@0"
I20260808 11:38:33.840441 5468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "1c51f9ff61e94208855cfb2586b890f8"
format_stamp: "Formatted at 2026-08-08 11:38:33 on dist-test-slave-tc2s"
server_key: "68d0d655c682fe3d73d8e2d8240ecb95"
server_key_iv: "b26d0490eaea548bef67eac8f4c340dc"
server_key_version: "encryptionkey@0"
I20260808 11:38:33.845470 5468 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260808 11:38:33.848757 5497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:33.850034 5468 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:38:33.850195 5468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "1c51f9ff61e94208855cfb2586b890f8"
format_stamp: "Formatted at 2026-08-08 11:38:33 on dist-test-slave-tc2s"
server_key: "68d0d655c682fe3d73d8e2d8240ecb95"
server_key_iv: "b26d0490eaea548bef67eac8f4c340dc"
server_key_version: "encryptionkey@0"
I20260808 11:38:33.850369 5468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:33.897264 5468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:33.901126 5468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:33.901401 5468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:33.918375 5468 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:44953
I20260808 11:38:33.918402 5573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:44953 every 8 connection(s)
I20260808 11:38:33.919683 5468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:38:33.923252 5574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:33.926324 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 5468
I20260808 11:38:33.926508 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:38:33.927171 24881 external_mini_cluster.cc:1428] Setting key 42fafc7feca8d41759f2c8f20e24e1bf
I20260808 11:38:33.935375 5574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8: Bootstrap starting.
I20260808 11:38:33.938967 5574 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:33.940213 5574 log.cc:824] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8: Log is configured to *not* fsync() on all Append() calls
Aug 08 11:38:33 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:38:33.943073 5574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8: No bootstrap required, opened a new log
I20260808 11:38:33.949374 5580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:33.929069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38800 (local address 127.24.76.126:44953)
0808 11:38:33.929832 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:33.929843 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:33.929885 (+ 42us) server_negotiation.cc:407] Connection header received
0808 11:38:33.931542 (+ 1657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:33.931564 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:33.931944 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:33.932247 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:33.934055 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:33.935308 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:33.937036 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:33.937721 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:33.937856 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:33.942527 (+ 4671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:33.942534 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:33.942549 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:33.942582 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:33.945493 (+ 2911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:33.946182 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:33.946192 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:33.946198 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:33.946380 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:33.946745 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:33.946749 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:33.946751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:33.947283 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:33.947591 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:33.948046 (+ 455us) server_negotiation.cc:299] Negotiation successful
0808 11:38:33.948413 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":336,"threads_started":1}
I20260808 11:38:33.950692 5574 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c51f9ff61e94208855cfb2586b890f8" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44953 } }
I20260808 11:38:33.951054 5574 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:33.951151 5574 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c51f9ff61e94208855cfb2586b890f8, State: Initialized, Role: FOLLOWER
I20260808 11:38:33.951710 5574 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c51f9ff61e94208855cfb2586b890f8" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44953 } }
I20260808 11:38:33.951987 5574 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:38:33.952127 5574 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:38:33.952268 5574 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:33.954334 5574 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c51f9ff61e94208855cfb2586b890f8" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44953 } }
I20260808 11:38:33.954767 5574 leader_election.cc:302] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c51f9ff61e94208855cfb2586b890f8; no voters:
I20260808 11:38:33.955271 5574 leader_election.cc:288] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:38:33.955379 5582 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:38:33.955957 5582 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8 [term 1 LEADER]: Becoming Leader. State: Replica: 1c51f9ff61e94208855cfb2586b890f8, State: Running, Role: LEADER
I20260808 11:38:33.956462 5582 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c51f9ff61e94208855cfb2586b890f8" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44953 } }
I20260808 11:38:33.957329 5574 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:38:33.959033 5584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c51f9ff61e94208855cfb2586b890f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c51f9ff61e94208855cfb2586b890f8" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44953 } } }
I20260808 11:38:33.959431 5584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8 [sys.catalog]: This master's current role is: LEADER
I20260808 11:38:33.960079 5585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c51f9ff61e94208855cfb2586b890f8. Latest consensus state: current_term: 1 leader_uuid: "1c51f9ff61e94208855cfb2586b890f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c51f9ff61e94208855cfb2586b890f8" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44953 } } }
I20260808 11:38:33.960383 5585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8 [sys.catalog]: This master's current role is: LEADER
I20260808 11:38:33.962776 5595 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:38:33.967854 5595 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:38:33.976584 5595 catalog_manager.cc:1347] Generated new cluster ID: b73ee9e988c948f39e74143d196db7cc
I20260808 11:38:33.976693 5595 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:38:33.984091 5595 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:38:33.985826 5595 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:38:33.992461 5595 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8: Generated new TSK 0
I20260808 11:38:33.993361 5595 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab.
Entry for principal kudu/127.24.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab.
I20260808 11:38:34.062392 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.65:0
--local_ip_for_outbound_sockets=127.24.76.65
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:44953
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:34965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:38:34.390367 5613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:34.390630 5613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:34.390765 5613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:34.397609 5613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:34.397713 5613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:34.397867 5613 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.65
I20260808 11:38:34.407166 5613 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:34965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:44953
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.65
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:34.408282 5613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:34.409523 5613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:34.422235 5619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:34.422192 5620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:34.422621 5622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:34.423007 5613 server_base.cc:1034] running on GCE node
I20260808 11:38:34.423549 5613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:34.424352 5613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:34.425550 5613 hybrid_clock.cc:648] HybridClock initialized: now 1786189114425514 us; error 30 us; skew 500 ppm
I20260808 11:38:34.425812 5613 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:38:34 dist-test-slave-tc2s krb5kdc[5451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189114, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:34.428731 5613 init.cc:377] Logged in from keytab as kudu/127.24.76.65@KRBTEST.COM (short username kudu)
I20260808 11:38:34.430130 5613 webserver.cc:460] Webserver started at http://127.24.76.65:45437/ using document root <none> and password file <none>
I20260808 11:38:34.430756 5613 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:34.430869 5613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:34.431152 5613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:34.433923 5613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-0/data/instance:
uuid: "8e2098352c05422da0a06a74d78e130a"
format_stamp: "Formatted at 2026-08-08 11:38:34 on dist-test-slave-tc2s"
server_key: "3c1f84b0b209b9d7aed32d9a2f8138b7"
server_key_iv: "503319b7711f4ac12d028ff049e08068"
server_key_version: "encryptionkey@0"
I20260808 11:38:34.434695 5613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance:
uuid: "8e2098352c05422da0a06a74d78e130a"
format_stamp: "Formatted at 2026-08-08 11:38:34 on dist-test-slave-tc2s"
server_key: "3c1f84b0b209b9d7aed32d9a2f8138b7"
server_key_iv: "503319b7711f4ac12d028ff049e08068"
server_key_version: "encryptionkey@0"
I20260808 11:38:34.439673 5613 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 11:38:34.442968 5632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:34.444151 5613 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 11:38:34.444293 5613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-0/wal
uuid: "8e2098352c05422da0a06a74d78e130a"
format_stamp: "Formatted at 2026-08-08 11:38:34 on dist-test-slave-tc2s"
server_key: "3c1f84b0b209b9d7aed32d9a2f8138b7"
server_key_iv: "503319b7711f4ac12d028ff049e08068"
server_key_version: "encryptionkey@0"
I20260808 11:38:34.444489 5613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:34.463441 5613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:34.467504 5613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:34.467775 5613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:34.468674 5613 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:38:34.470059 5613 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:38:34.470146 5613 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:34.470273 5613 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:38:34.470324 5613 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:34.499738 5613 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.65:36707
I20260808 11:38:34.499782 5807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.65:36707 every 8 connection(s)
I20260808 11:38:34.500954 5613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
I20260808 11:38:34.506469 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 5613
I20260808 11:38:34.506644 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance
I20260808 11:38:34.507256 24881 external_mini_cluster.cc:1428] Setting key 1635ae9a982393fd84f907b005ab129d
Aug 08 11:38:34 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189114, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:38:34.520668 5812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:34.503269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:48259 (local address 127.24.76.126:44953)
0808 11:38:34.503737 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:34.503744 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:34.504679 (+ 935us) server_negotiation.cc:407] Connection header received
0808 11:38:34.506703 (+ 2024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:34.506708 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:34.506806 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:34.506905 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:34.508541 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:34.509431 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:34.511009 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:34.511400 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:34.511515 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:34.514908 (+ 3393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:34.514913 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:34.514916 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:34.514948 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:34.517514 (+ 2566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:34.518340 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:34.518344 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:34.518347 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:34.518473 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:34.518895 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:34.518899 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:34.518902 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:34.519171 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:34.519299 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:34.520092 (+ 793us) server_negotiation.cc:299] Negotiation successful
0808 11:38:34.520334 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":228,"threads_started":1}
I20260808 11:38:34.521503 5813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:34.503798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.65:48259)
0808 11:38:34.504524 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:34.504572 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:34.506427 (+ 1855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:34.507160 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:34.507168 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:34.507698 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:34.508365 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:34.508387 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:34.509589 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:34.509594 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:34.510868 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:34.510881 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:34.511624 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:34.511631 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:34.511847 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:34.512445 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:34.512473 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:34.514700 (+ 2227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:34.517655 (+ 2955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:34.517663 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:34.517677 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:34.518236 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:34.518568 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:34.518572 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:34.518575 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:34.518800 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:34.519285 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:34.519292 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:34.519673 (+ 381us) client_negotiation.cc:769] Sending connection context
0808 11:38:34.519939 (+ 266us) client_negotiation.cc:240] Negotiation successful
0808 11:38:34.520427 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":254,"threads_started":1}
I20260808 11:38:34.522941 5810 heartbeater.cc:347] Connected to a master server at 127.24.76.126:44953
I20260808 11:38:34.523216 5810 heartbeater.cc:464] Registering TS with master...
I20260808 11:38:34.523878 5810 heartbeater.cc:511] Master 127.24.76.126:44953 requested a full tablet report, sending...
I20260808 11:38:34.526227 5519 ts_manager.cc:194] Registered new tserver with Master: 8e2098352c05422da0a06a74d78e130a (127.24.76.65:36707)
I20260808 11:38:34.528179 5519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.65@KRBTEST.COM'} at 127.24.76.65:48259
WARNING: no policy specified for kudu/127.24.76.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab.
Entry for principal kudu/127.24.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab.
I20260808 11:38:34.564673 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.66:0
--local_ip_for_outbound_sockets=127.24.76.66
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:44953
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:34965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:38:34.883602 5821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:34.883857 5821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:34.883972 5821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:34.890867 5821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:34.890965 5821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:34.891109 5821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.66
I20260808 11:38:34.900171 5821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:34965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:44953
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.66
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:34.901328 5821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:34.902616 5821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:34.914659 5835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:34.914613 5830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:34.915032 5829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:34.915354 5821 server_base.cc:1034] running on GCE node
I20260808 11:38:34.915912 5821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:34.916738 5821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:34.917963 5821 hybrid_clock.cc:648] HybridClock initialized: now 1786189114917912 us; error 40 us; skew 500 ppm
I20260808 11:38:34.918296 5821 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:38:34 dist-test-slave-tc2s krb5kdc[5451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189114, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:34.922072 5821 init.cc:377] Logged in from keytab as kudu/127.24.76.66@KRBTEST.COM (short username kudu)
I20260808 11:38:34.923691 5821 webserver.cc:460] Webserver started at http://127.24.76.66:38953/ using document root <none> and password file <none>
I20260808 11:38:34.924502 5821 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:34.924620 5821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:34.924923 5821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:34.928486 5821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-1/data/instance:
uuid: "993f6f7f5feb4edfa25ec24a7d27f14f"
format_stamp: "Formatted at 2026-08-08 11:38:34 on dist-test-slave-tc2s"
server_key: "92761cd875e97775d0b1eabd6f88386a"
server_key_iv: "0ab18bf88de2e5a3b9a645c9ed576d5d"
server_key_version: "encryptionkey@0"
I20260808 11:38:34.929378 5821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance:
uuid: "993f6f7f5feb4edfa25ec24a7d27f14f"
format_stamp: "Formatted at 2026-08-08 11:38:34 on dist-test-slave-tc2s"
server_key: "92761cd875e97775d0b1eabd6f88386a"
server_key_iv: "0ab18bf88de2e5a3b9a645c9ed576d5d"
server_key_version: "encryptionkey@0"
I20260808 11:38:34.935474 5821 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260808 11:38:34.940505 5846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:34.941717 5821 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260808 11:38:34.941864 5821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-1/wal
uuid: "993f6f7f5feb4edfa25ec24a7d27f14f"
format_stamp: "Formatted at 2026-08-08 11:38:34 on dist-test-slave-tc2s"
server_key: "92761cd875e97775d0b1eabd6f88386a"
server_key_iv: "0ab18bf88de2e5a3b9a645c9ed576d5d"
server_key_version: "encryptionkey@0"
I20260808 11:38:34.942101 5821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:34.961498 5821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:34.965075 5821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:34.965325 5821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:34.966231 5821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:38:34.967444 5821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:38:34.967528 5821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:34.967650 5821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:38:34.967703 5821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:34.998075 5821 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.66:37737
I20260808 11:38:34.998112 6022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.66:37737 every 8 connection(s)
I20260808 11:38:34.999256 5821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
I20260808 11:38:35.005688 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 5821
I20260808 11:38:35.005846 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance
I20260808 11:38:35.006295 24881 external_mini_cluster.cc:1428] Setting key b85c36f25fc35d5ffa9bc09745a21240
Aug 08 11:38:35 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189114, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:38:35.019132 5812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:35.001597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:36743 (local address 127.24.76.126:44953)
0808 11:38:35.001761 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:35.001765 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:35.002942 (+ 1177us) server_negotiation.cc:407] Connection header received
0808 11:38:35.004734 (+ 1792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:35.004739 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:35.004819 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:35.004909 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:35.006435 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:35.007282 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:35.009167 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:35.009594 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:35.009692 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:35.013363 (+ 3671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:35.013369 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:35.013373 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:35.013409 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:35.016275 (+ 2866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:35.016938 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:35.016942 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:35.016952 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:35.017097 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:35.017547 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:35.017551 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:35.017553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:35.017866 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:35.018040 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:35.018615 (+ 575us) server_negotiation.cc:299] Negotiation successful
0808 11:38:35.018819 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 11:38:35.019865 6025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:35.002103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.66:36743)
0808 11:38:35.002783 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:35.002827 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:35.004476 (+ 1649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:35.005156 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:35.005164 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:35.005650 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:35.006264 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:35.006283 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:35.007432 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:35.007436 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:35.009021 (+ 1585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:35.009038 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:35.009813 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:35.009820 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:35.010130 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:35.010796 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:35.010829 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:35.013150 (+ 2321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:35.016399 (+ 3249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:35.016405 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:35.016416 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:35.016816 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:35.017205 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:35.017210 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:35.017213 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:35.017437 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:35.017997 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:35.018003 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:35.018284 (+ 281us) client_negotiation.cc:769] Sending connection context
0808 11:38:35.018503 (+ 219us) client_negotiation.cc:240] Negotiation successful
0808 11:38:35.018841 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":270,"threads_started":1}
I20260808 11:38:35.021278 6023 heartbeater.cc:347] Connected to a master server at 127.24.76.126:44953
I20260808 11:38:35.021579 6023 heartbeater.cc:464] Registering TS with master...
I20260808 11:38:35.022382 6023 heartbeater.cc:511] Master 127.24.76.126:44953 requested a full tablet report, sending...
I20260808 11:38:35.023794 5519 ts_manager.cc:194] Registered new tserver with Master: 993f6f7f5feb4edfa25ec24a7d27f14f (127.24.76.66:37737)
I20260808 11:38:35.024536 5519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.66@KRBTEST.COM'} at 127.24.76.66:36743
WARNING: no policy specified for kudu/127.24.76.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab.
Entry for principal kudu/127.24.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab.
I20260808 11:38:35.062623 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.67:0
--local_ip_for_outbound_sockets=127.24.76.67
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:44953
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:34965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:38:35.386010 6028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:35.386332 6028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:35.386493 6028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:35.393446 6028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:35.393553 6028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:35.393748 6028 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.67
I20260808 11:38:35.403997 6028 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:34965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:44953
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.67
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:35.405522 6028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:35.407302 6028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:35.421072 6033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:35.421046 6043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:35.421103 6035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:35.421811 6028 server_base.cc:1034] running on GCE node
I20260808 11:38:35.422469 6028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:35.423309 6028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:35.424503 6028 hybrid_clock.cc:648] HybridClock initialized: now 1786189115424457 us; error 38 us; skew 500 ppm
I20260808 11:38:35.424791 6028 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:38:35 dist-test-slave-tc2s krb5kdc[5451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189115, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:35.427754 6028 init.cc:377] Logged in from keytab as kudu/127.24.76.67@KRBTEST.COM (short username kudu)
I20260808 11:38:35.429162 6028 webserver.cc:460] Webserver started at http://127.24.76.67:39671/ using document root <none> and password file <none>
I20260808 11:38:35.429800 6028 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:35.429931 6028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:35.430222 6028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:35.432998 6028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-2/data/instance:
uuid: "c440772fcc2e42d68bc13c28b3e13768"
format_stamp: "Formatted at 2026-08-08 11:38:35 on dist-test-slave-tc2s"
server_key: "e3c21d419c52529223ad9bda1678a2a3"
server_key_iv: "90f8a88129cc5aa29f9017de677505d6"
server_key_version: "encryptionkey@0"
I20260808 11:38:35.433760 6028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance:
uuid: "c440772fcc2e42d68bc13c28b3e13768"
format_stamp: "Formatted at 2026-08-08 11:38:35 on dist-test-slave-tc2s"
server_key: "e3c21d419c52529223ad9bda1678a2a3"
server_key_iv: "90f8a88129cc5aa29f9017de677505d6"
server_key_version: "encryptionkey@0"
I20260808 11:38:35.438733 6028 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 11:38:35.441885 6053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:35.443090 6028 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 11:38:35.443231 6028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-2/wal
uuid: "c440772fcc2e42d68bc13c28b3e13768"
format_stamp: "Formatted at 2026-08-08 11:38:35 on dist-test-slave-tc2s"
server_key: "e3c21d419c52529223ad9bda1678a2a3"
server_key_iv: "90f8a88129cc5aa29f9017de677505d6"
server_key_version: "encryptionkey@0"
I20260808 11:38:35.443403 6028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:35.468099 6028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:35.472251 6028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:35.472533 6028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:35.473474 6028 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:38:35.474807 6028 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:38:35.474886 6028 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:35.475018 6028 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:38:35.475071 6028 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:35.505707 6028 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.67:46685
I20260808 11:38:35.505759 6229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.67:46685 every 8 connection(s)
I20260808 11:38:35.506891 6028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
I20260808 11:38:35.514216 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 6028
I20260808 11:38:35.514362 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance
I20260808 11:38:35.514806 24881 external_mini_cluster.cc:1428] Setting key c9e8376bb67878b80987b1f03c528889
Aug 08 11:38:35 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189115, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:38:35.525579 5812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:35.509165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:41035 (local address 127.24.76.126:44953)
0808 11:38:35.509394 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:35.509410 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:35.510482 (+ 1072us) server_negotiation.cc:407] Connection header received
0808 11:38:35.512214 (+ 1732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:35.512218 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:35.512314 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:35.512400 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:35.513720 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:35.514546 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:35.516306 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:35.516839 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:35.516950 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:35.520214 (+ 3264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:35.520220 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:35.520225 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:35.520265 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:35.522753 (+ 2488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:35.523427 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:35.523432 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:35.523436 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:35.523566 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:35.523975 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:35.523979 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:35.523981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:35.524248 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:35.524388 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:35.525019 (+ 631us) server_negotiation.cc:299] Negotiation successful
0808 11:38:35.525233 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 11:38:35.526453 6232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:35.509639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.67:41035)
0808 11:38:35.510340 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:35.510377 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:35.511979 (+ 1602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:35.512569 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:35.512576 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:35.513065 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:35.513587 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:35.513605 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:35.514705 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:35.514709 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:35.516161 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:35.516174 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:35.517066 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:35.517074 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:35.517289 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:35.517923 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:35.517953 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:35.520010 (+ 2057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:35.522881 (+ 2871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:35.522890 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:35.522912 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:35.523305 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:35.523671 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:38:35.523676 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:38:35.523680 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:38:35.523858 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:38:35.524385 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:35.524393 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:35.524679 (+ 286us) client_negotiation.cc:769] Sending connection context
0808 11:38:35.524913 (+ 234us) client_negotiation.cc:240] Negotiation successful
0808 11:38:35.525376 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":302,"threads_started":1}
I20260808 11:38:35.527666 6230 heartbeater.cc:347] Connected to a master server at 127.24.76.126:44953
I20260808 11:38:35.527959 6230 heartbeater.cc:464] Registering TS with master...
I20260808 11:38:35.528697 6230 heartbeater.cc:511] Master 127.24.76.126:44953 requested a full tablet report, sending...
I20260808 11:38:35.530118 5519 ts_manager.cc:194] Registered new tserver with Master: c440772fcc2e42d68bc13c28b3e13768 (127.24.76.67:46685)
I20260808 11:38:35.530898 5519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.67@KRBTEST.COM'} at 127.24.76.67:41035
I20260808 11:38:35.530920 5810 heartbeater.cc:503] Master 127.24.76.126:44953 was elected leader, sending a full tablet report...
I20260808 11:38:35.541471 24881 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.24.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.65.keytab.
Entry for principal oryx/127.24.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.65.keytab.
WARNING: no policy specified for oryx/127.24.76.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.76.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.66.keytab.
Entry for principal oryx/127.24.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.66.keytab.
WARNING: no policy specified for oryx/127.24.76.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.76.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.67.keytab.
Entry for principal oryx/127.24.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.67.keytab.
I20260808 11:38:35.686766 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 5613
Aug 08 11:38:35 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.65@KRBTEST.COM
I20260808 11:38:35.702675 6245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:35.687873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40222 (local address 127.24.76.65:36707)
0808 11:38:35.688516 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:35.688525 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:35.688560 (+ 35us) server_negotiation.cc:407] Connection header received
0808 11:38:35.688674 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:35.688682 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:35.688906 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:35.689052 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:35.690524 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:35.691555 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:35.693224 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:35.693801 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:35.693937 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:35.697821 (+ 3884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:35.697833 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:35.697850 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:35.697916 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:35.700399 (+ 2483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:35.700989 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:35.700996 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:35.700998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:35.701116 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:35.701384 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:35.701387 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:35.701389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:35.701733 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:35.702000 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:35.702182 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 11:38:35.702381 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":277,"threads_started":1}
I20260808 11:38:35.703931 5800 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:35.802130 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 5613
I20260808 11:38:35.826673 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 5821
Aug 08 11:38:35 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.66@KRBTEST.COM
I20260808 11:38:35.841590 6247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:35.827573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41946 (local address 127.24.76.66:37737)
0808 11:38:35.828137 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:35.828144 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:35.828170 (+ 26us) server_negotiation.cc:407] Connection header received
0808 11:38:35.828240 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:35.828247 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:35.828444 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:35.828574 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:35.829744 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:35.830805 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:35.831974 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:35.832511 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:35.832602 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:35.836635 (+ 4033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:35.836648 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:35.836663 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:35.836701 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:35.839279 (+ 2578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:35.839871 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:35.839877 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:35.839880 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:35.840013 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:35.840275 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:35.840277 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:35.840279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:35.840615 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:35.840851 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:35.841100 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 11:38:35.841298 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":291,"threads_started":1}
I20260808 11:38:35.842895 6017 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:35.923519 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 5821
I20260808 11:38:35.942219 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 6028
Aug 08 11:38:35 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.67@KRBTEST.COM
I20260808 11:38:35.958127 6250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:35.943160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45852 (local address 127.24.76.67:46685)
0808 11:38:35.943651 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:35.943658 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:35.943689 (+ 31us) server_negotiation.cc:407] Connection header received
0808 11:38:35.943750 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:35.943756 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:35.943970 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:35.944117 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:35.945456 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:35.946484 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:35.947755 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:35.948333 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:35.948455 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:35.952706 (+ 4251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:35.952717 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:35.952730 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:35.952762 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:35.955466 (+ 2704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:35.956081 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:35.956088 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:35.956091 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:35.956210 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:35.956553 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:35.956557 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:35.956559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:35.956888 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:35.957171 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:35.957468 (+ 297us) server_negotiation.cc:299] Negotiation successful
0808 11:38:35.957718 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":206,"threads_started":1}
I20260808 11:38:35.959318 6223 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:36.040637 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 6028
I20260808 11:38:36.061550 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 5468
I20260808 11:38:36.062740 5569 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:36.164294 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 5468
I20260808 11:38:36.189533 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:44953
--webserver_interface=127.24.76.126
--webserver_port=38821
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:34965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:44953
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:38:36.515681 6256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:36.515946 6256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:36.516039 6256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:36.522953 6256 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:38:36.523046 6256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:36.523097 6256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:36.523152 6256 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:38:36.523195 6256 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:38:36.532428 6256 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:34965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:44953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:44953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=38821
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:36.533564 6256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:36.534789 6256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:36.545742 6266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:36.545807 6265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:36.545912 6271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:36.546528 6256 server_base.cc:1034] running on GCE node
I20260808 11:38:36.547255 6256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:36.548509 6256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:36.549700 6256 hybrid_clock.cc:648] HybridClock initialized: now 1786189116549643 us; error 47 us; skew 500 ppm
I20260808 11:38:36.550062 6256 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:38:36 dist-test-slave-tc2s krb5kdc[5451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189116, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:36.552982 6256 init.cc:377] Logged in from keytab as kudu/127.24.76.126@KRBTEST.COM (short username kudu)
I20260808 11:38:36.554340 6256 webserver.cc:460] Webserver started at http://127.24.76.126:38821/ using document root <none> and password file <none>
I20260808 11:38:36.554988 6256 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:36.555116 6256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:36.560235 6256 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 11:38:36.562917 6279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:36.564092 6256 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:38:36.564236 6256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "1c51f9ff61e94208855cfb2586b890f8"
format_stamp: "Formatted at 2026-08-08 11:38:33 on dist-test-slave-tc2s"
server_key: "68d0d655c682fe3d73d8e2d8240ecb95"
server_key_iv: "b26d0490eaea548bef67eac8f4c340dc"
server_key_version: "encryptionkey@0"
I20260808 11:38:36.565176 6256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:36.601801 6256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:36.605578 6256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:36.605825 6256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:36.622731 6256 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:44953
I20260808 11:38:36.622751 6361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:44953 every 8 connection(s)
I20260808 11:38:36.624150 6256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:38:36.630756 6362 sys_catalog.cc:263] Verifying existing consensus state
I20260808 11:38:36.632977 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 6256
I20260808 11:38:36.633755 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.65:36707
--local_ip_for_outbound_sockets=127.24.76.65
--tserver_master_addrs=127.24.76.126:44953
--webserver_port=45437
--webserver_interface=127.24.76.65
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:34965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.76.65
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:38:36.633911 6362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8: Bootstrap starting.
I20260808 11:38:36.652535 6362 log.cc:824] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8: Log is configured to *not* fsync() on all Append() calls
I20260808 11:38:36.661069 6362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 11:38:36.661558 6362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8: Bootstrap complete.
I20260808 11:38:36.670652 6362 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c51f9ff61e94208855cfb2586b890f8" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44953 } }
I20260808 11:38:36.671473 6362 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c51f9ff61e94208855cfb2586b890f8, State: Initialized, Role: FOLLOWER
I20260808 11:38:36.672075 6362 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c51f9ff61e94208855cfb2586b890f8" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44953 } }
I20260808 11:38:36.672302 6362 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:38:36.672407 6362 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:38:36.672569 6362 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8 [term 1 FOLLOWER]: Advancing to term 2
I20260808 11:38:36.676410 6362 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c51f9ff61e94208855cfb2586b890f8" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44953 } }
I20260808 11:38:36.676860 6362 leader_election.cc:302] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c51f9ff61e94208855cfb2586b890f8; no voters:
I20260808 11:38:36.677438 6362 leader_election.cc:288] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 11:38:36.677582 6367 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 11:38:36.678759 6367 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8 [term 2 LEADER]: Becoming Leader. State: Replica: 1c51f9ff61e94208855cfb2586b890f8, State: Running, Role: LEADER
I20260808 11:38:36.679378 6367 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c51f9ff61e94208855cfb2586b890f8" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44953 } }
I20260808 11:38:36.679545 6362 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:38:36.681823 6369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c51f9ff61e94208855cfb2586b890f8. Latest consensus state: current_term: 2 leader_uuid: "1c51f9ff61e94208855cfb2586b890f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c51f9ff61e94208855cfb2586b890f8" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44953 } } }
I20260808 11:38:36.682209 6369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8 [sys.catalog]: This master's current role is: LEADER
I20260808 11:38:36.682726 6368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1c51f9ff61e94208855cfb2586b890f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c51f9ff61e94208855cfb2586b890f8" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44953 } } }
I20260808 11:38:36.683017 6368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8 [sys.catalog]: This master's current role is: LEADER
I20260808 11:38:36.693177 6375 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:38:36.699220 6375 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:38:36.700614 6375 catalog_manager.cc:1259] Loaded cluster ID: b73ee9e988c948f39e74143d196db7cc
I20260808 11:38:36.700762 6375 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:38:36.704555 6375 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:38:36.706001 6375 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1c51f9ff61e94208855cfb2586b890f8: Loaded TSK: 0
I20260808 11:38:36.707532 6375 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 11:38:36.973869 6364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:36.974197 6364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:36.974339 6364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:36.981957 6364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:36.982074 6364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:36.982239 6364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.65
I20260808 11:38:36.991710 6364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:34965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.65.keytab
--principal=oryx/127.24.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.65:36707
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.76.65
--webserver_port=45437
--tserver_master_addrs=127.24.76.126:44953
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.65
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:36.992955 6364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:36.994282 6364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:37.008152 6404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:37.008838 6406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:37.008980 6403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:37.009601 6364 server_base.cc:1034] running on GCE node
I20260808 11:38:37.010187 6364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:37.011118 6364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:37.012331 6364 hybrid_clock.cc:648] HybridClock initialized: now 1786189117012285 us; error 41 us; skew 500 ppm
I20260808 11:38:37.012650 6364 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:38:37 dist-test-slave-tc2s krb5kdc[5451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189117, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:37.019116 6364 init.cc:377] Logged in from keytab as oryx/127.24.76.65@KRBTEST.COM (short username oryx)
I20260808 11:38:37.020569 6364 webserver.cc:460] Webserver started at http://127.24.76.65:45437/ using document root <none> and password file <none>
I20260808 11:38:37.021209 6364 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:37.021339 6364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:37.026468 6364 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:38:37.029009 6419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:37.030280 6364 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:38:37.030416 6364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-0/wal
uuid: "8e2098352c05422da0a06a74d78e130a"
format_stamp: "Formatted at 2026-08-08 11:38:34 on dist-test-slave-tc2s"
server_key: "3c1f84b0b209b9d7aed32d9a2f8138b7"
server_key_iv: "503319b7711f4ac12d028ff049e08068"
server_key_version: "encryptionkey@0"
I20260808 11:38:37.031312 6364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:37.057931 6364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:37.061844 6364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:37.062105 6364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:37.062981 6364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:38:37.064236 6364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:38:37.064327 6364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:37.064426 6364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:38:37.064507 6364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:37.094107 6364 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.65:36707
I20260808 11:38:37.094213 6596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.65:36707 every 8 connection(s)
I20260808 11:38:37.095497 6364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
I20260808 11:38:37.100046 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 6364
I20260808 11:38:37.100647 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.66:37737
--local_ip_for_outbound_sockets=127.24.76.66
--tserver_master_addrs=127.24.76.126:44953
--webserver_port=38953
--webserver_interface=127.24.76.66
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:34965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.76.66
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 11:38:37 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:37 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 11:38:37.432438 6605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:37.432701 6605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:37.432830 6605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:37.439952 6605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:37.440065 6605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:37.440205 6605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.66
I20260808 11:38:37.449772 6605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:34965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.66.keytab
--principal=oryx/127.24.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.66:37737
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.76.66
--webserver_port=38953
--tserver_master_addrs=127.24.76.126:44953
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.66
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:37.451009 6605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:37.452246 6605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:37.467105 6615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:37.467535 6614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:37.467732 6619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:37.468194 6605 server_base.cc:1034] running on GCE node
I20260808 11:38:37.468837 6605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:37.469820 6605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:37.471035 6605 hybrid_clock.cc:648] HybridClock initialized: now 1786189117470970 us; error 63 us; skew 500 ppm
I20260808 11:38:37.471325 6605 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:38:37 dist-test-slave-tc2s krb5kdc[5451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189117, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:37.474552 6605 init.cc:377] Logged in from keytab as oryx/127.24.76.66@KRBTEST.COM (short username oryx)
I20260808 11:38:37.476018 6605 webserver.cc:460] Webserver started at http://127.24.76.66:38953/ using document root <none> and password file <none>
I20260808 11:38:37.476665 6605 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:37.476784 6605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:37.481973 6605 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:38:37.484444 6627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:37.485679 6605 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 11:38:37.485819 6605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-1/wal
uuid: "993f6f7f5feb4edfa25ec24a7d27f14f"
format_stamp: "Formatted at 2026-08-08 11:38:34 on dist-test-slave-tc2s"
server_key: "92761cd875e97775d0b1eabd6f88386a"
server_key_iv: "0ab18bf88de2e5a3b9a645c9ed576d5d"
server_key_version: "encryptionkey@0"
I20260808 11:38:37.486757 6605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:37.504626 6605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:37.508251 6605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:37.508483 6605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:37.509424 6605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:38:37.510792 6605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:38:37.510887 6605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:37.510994 6605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:38:37.511059 6605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:37.550949 6605 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.66:37737
I20260808 11:38:37.551000 6814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.66:37737 every 8 connection(s)
I20260808 11:38:37.552325 6605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
I20260808 11:38:37.561136 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 6605
I20260808 11:38:37.562007 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.67:46685
--local_ip_for_outbound_sockets=127.24.76.67
--tserver_master_addrs=127.24.76.126:44953
--webserver_port=39671
--webserver_interface=127.24.76.67
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:34965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.76.67
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 11:38:37 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:37 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 11:38:37.924029 6819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:37.924286 6819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:37.924451 6819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:37.931619 6819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:37.931733 6819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:37.931893 6819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.67
I20260808 11:38:37.941736 6819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:34965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/krb5kdc/oryx_127.24.76.67.keytab
--principal=oryx/127.24.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.67:46685
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.76.67
--webserver_port=39671
--tserver_master_addrs=127.24.76.126:44953
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.67
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:37.942994 6819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:37.944366 6819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:37.958966 6826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:37.962299 6827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:37.967561 6830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:37.967981 6819 server_base.cc:1034] running on GCE node
I20260808 11:38:37.968809 6819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:37.974120 6819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:37.975360 6819 hybrid_clock.cc:648] HybridClock initialized: now 1786189117975301 us; error 54 us; skew 500 ppm
I20260808 11:38:37.975672 6819 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:38:37 dist-test-slave-tc2s krb5kdc[5451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189117, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:37.978929 6819 init.cc:377] Logged in from keytab as oryx/127.24.76.67@KRBTEST.COM (short username oryx)
I20260808 11:38:37.980717 6819 webserver.cc:460] Webserver started at http://127.24.76.67:39671/ using document root <none> and password file <none>
I20260808 11:38:37.981477 6819 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:37.981612 6819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:37.987007 6819 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 11:38:37.989538 6842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:37.990955 6819 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 11:38:37.991106 6819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-2/wal
uuid: "c440772fcc2e42d68bc13c28b3e13768"
format_stamp: "Formatted at 2026-08-08 11:38:35 on dist-test-slave-tc2s"
server_key: "e3c21d419c52529223ad9bda1678a2a3"
server_key_iv: "90f8a88129cc5aa29f9017de677505d6"
server_key_version: "encryptionkey@0"
I20260808 11:38:37.992013 6819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:38.018023 6819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:38.021965 6819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:38.022226 6819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:38.023205 6819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:38:38.024585 6819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:38:38.024679 6819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:38.024785 6819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:38:38.024859 6819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:38.064455 6819 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.67:46685
I20260808 11:38:38.064530 7015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.67:46685 every 8 connection(s)
I20260808 11:38:38.065937 6819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
I20260808 11:38:38.066651 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 6819
I20260808 11:38:38.120101 7019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:38.067776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38806 (local address 127.24.76.126:44953)
0808 11:38:38.088012 (+ 20236us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:38.088019 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:38.088046 (+ 27us) server_negotiation.cc:407] Connection header received
0808 11:38:38.088128 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:38.088134 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:38.088223 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:38.088378 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:38.098015 (+ 9637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:38.099415 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:38.105089 (+ 5674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:38.105859 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:38.106015 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:38.109352 (+ 3337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:38.109374 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:38.109399 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:38.109447 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:38.113938 (+ 4491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:38.114955 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:38.114966 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:38.114976 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:38.115263 (+ 287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:38:38.115789 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:38:38.115796 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:38:38.115800 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:38:38.116506 (+ 706us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:38.118142 (+ 1636us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:38.118463 (+ 321us) server_negotiation.cc:299] Negotiation successful
0808 11:38:38.118969 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20082,"thread_start_us":20832,"threads_started":1}
Aug 08 11:38:38 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:38 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 11:38:38.445302 6600 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.337s user 0.000s sys 0.016s
W20260808 11:38:38.446990 6599 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:37.098239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:54353 (local address 127.24.76.126:44953)
0808 11:38:37.098873 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:37.098884 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:37.099175 (+ 291us) server_negotiation.cc:407] Connection header received
0808 11:38:37.101389 (+ 2214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:37.101407 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:37.101831 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:37.102242 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:37.103505 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:37.105131 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:37.106600 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:37.107497 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:37.107662 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:38.446661 (+1338999us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.65:54353: BlockingRecv error: recv got EOF from 127.24.76.65:54353 (error 108)
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":239,"threads_started":1}
I20260808 11:38:38.447731 6600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:37.098254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.65:54353)
0808 11:38:37.099026 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:37.099071 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:37.100812 (+ 1741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:37.102319 (+ 1507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:37.102326 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:37.102798 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:37.103311 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:37.103328 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:37.105250 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:37.105253 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:37.106466 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:37.106478 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:37.107673 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:37.107677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:37.107861 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:37.108459 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:37.108479 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:38.446413 (+1337934us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":318,"threads_started":1}
W20260808 11:38:38.448704 6597 heartbeater.cc:644] Failed to heartbeat to 127.24.76.126:44953 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.76.126:44953: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Aug 08 11:38:38 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:38 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:38.466113 6600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:38.449509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.65:39297)
0808 11:38:38.449973 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:38.450005 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:38.450138 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:38.450647 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:38.450653 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:38.451182 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:38.451440 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:38.451452 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:38.454874 (+ 3422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:38.454880 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:38.456572 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:38.456589 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:38.458031 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:38.458040 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:38.458252 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:38.460131 (+ 1879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:38.460158 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:38.465741 (+ 5583us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":281}
W20260808 11:38:38.466384 6599 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:38.450066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:39297 (local address 127.24.76.126:44953)
0808 11:38:38.450264 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:38.450271 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:38.450293 (+ 22us) server_negotiation.cc:407] Connection header received
0808 11:38:38.450368 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:38.450375 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:38.450461 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:38.450580 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:38.453352 (+ 2772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:38.454703 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:38.456776 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:38.457512 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:38.457624 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:38.466085 (+ 8461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.65:39297: BlockingRecv error: recv got EOF from 127.24.76.65:39297 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 08 11:38:38 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:38 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 11:38:38.486235 6599 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:38.467708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:56719 (local address 127.24.76.126:44953)
0808 11:38:38.467878 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:38.467884 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:38.473285 (+ 5401us) server_negotiation.cc:407] Connection header received
0808 11:38:38.473735 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:38.473742 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:38.473858 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:38.473950 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:38.475080 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:38.476362 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:38.478357 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:38.479061 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:38.479190 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:38.485925 (+ 6735us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.65:56719: BlockingRecv error: recv got EOF from 127.24.76.65:56719 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 11:38:38.486928 6600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:38.467269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.65:56719)
0808 11:38:38.473170 (+ 5901us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:38.473193 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:38.473325 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:38.474195 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:38.474201 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:38.474654 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:38.474888 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:38.474900 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:38.476529 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:38.476535 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:38.478215 (+ 1680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:38.478231 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:38.479223 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:38.479230 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:38.479429 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:38.481166 (+ 1737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:38.481195 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:38.486593 (+ 5398us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":5786,"thread_start_us":6174,"threads_started":1}
W20260808 11:38:38.487340 6597 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 11:38:38.726019 6818 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.162s user 0.001s sys 0.014s
W20260808 11:38:38.727473 6817 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:37.555019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:53889 (local address 127.24.76.126:44953)
0808 11:38:37.555458 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:37.555462 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:37.556753 (+ 1291us) server_negotiation.cc:407] Connection header received
0808 11:38:37.558592 (+ 1839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:37.558598 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:37.558681 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:37.558780 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:37.560232 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:37.560997 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:37.562751 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:37.563158 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:37.563274 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:38.727240 (+1163966us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.66:53889: BlockingRecv error: recv got EOF from 127.24.76.66:53889 (error 108)
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":223,"threads_started":1}
I20260808 11:38:38.728609 6818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:37.555798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.66:53889)
0808 11:38:37.556596 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:37.556639 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:37.558326 (+ 1687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:37.559021 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:37.559030 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:37.559523 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:37.560062 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:37.560080 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:37.561151 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:37.561155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:37.562616 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:37.562629 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:37.563372 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:37.563379 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:37.563619 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:37.564213 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:37.564235 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:38.727113 (+1162878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":323,"threads_started":1}
W20260808 11:38:38.729617 6815 heartbeater.cc:644] Failed to heartbeat to 127.24.76.126:44953 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.76.126:44953: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Aug 08 11:38:38 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:38 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 11:38:38.740892 6817 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:38.730350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:40929 (local address 127.24.76.126:44953)
0808 11:38:38.730501 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:38.730506 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:38.731032 (+ 526us) server_negotiation.cc:407] Connection header received
0808 11:38:38.731246 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:38.731249 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:38.731313 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:38.731394 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:38.732435 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:38.733776 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:38.735199 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:38.735886 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:38.736019 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:38.740723 (+ 4704us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.66:40929: BlockingRecv error: recv got EOF from 127.24.76.66:40929 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 11:38:38.740918 6818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:38.730236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.66:40929)
0808 11:38:38.730935 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:38.730959 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:38.731111 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:38.731453 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:38.731459 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:38.731920 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:38.732119 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:38.732134 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:38.733986 (+ 1852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:38.733992 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:38.735030 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:38.735046 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:38.736060 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:38.736065 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:38.736183 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:38.736849 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:38.736866 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:38.740579 (+ 3713us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":526}
Aug 08 11:38:38 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:38 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:38.750506 6818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:38.741647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.66:33535)
0808 11:38:38.741788 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:38.741803 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:38.741873 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:38.742436 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:38.742440 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:38.742704 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:38.742850 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:38.742858 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:38.744408 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:38.744413 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:38.745637 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:38.745649 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:38.746610 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:38.746614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:38.746733 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:38.747283 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:38.747299 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:38.750322 (+ 3023us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 11:38:38.750770 6815 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 11:38:38.750871 6817 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:38.741939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:33535 (local address 127.24.76.126:44953)
0808 11:38:38.742112 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:38.742118 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:38.742137 (+ 19us) server_negotiation.cc:407] Connection header received
0808 11:38:38.742201 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:38.742207 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:38.742280 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:38.742383 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:38.742979 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:38.744212 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:38.745772 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:38.746466 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:38.746600 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:38.750589 (+ 3989us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.66:33535: BlockingRecv error: recv got EOF from 127.24.76.66:33535 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260808 11:38:39.148975 7022 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.028s user 0.006s sys 0.015s
W20260808 11:38:39.150125 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:38.084135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:33303 (local address 127.24.76.126:44953)
0808 11:38:38.105605 (+ 21470us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:38.105609 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:38.106805 (+ 1196us) server_negotiation.cc:407] Connection header received
0808 11:38:38.109303 (+ 2498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:38.109309 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:38.109391 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:38.109496 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:38.114049 (+ 4553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:38.115469 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:38.118149 (+ 2680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:38.118869 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:38.118977 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:39.149885 (+1030908us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.67:33303: BlockingRecv error: recv got EOF from 127.24.76.67:33303 (error 108)
Metrics: {"server-negotiator.queue_time_us":21347,"thread_start_us":21465,"threads_started":1}
I20260808 11:38:39.151615 7022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:38.083385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.67:33303)
0808 11:38:38.106176 (+ 22791us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:38.106236 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:38.108972 (+ 2736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:38.110031 (+ 1059us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:38.110043 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:38.110849 (+ 806us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:38.111729 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:38.111757 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:38.115693 (+ 3936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:38.115700 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:38.117959 (+ 2259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:38.117978 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:38.119403 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:38.119410 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:38.119736 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:38.120740 (+ 1004us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:38.120774 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:39.149710 (+1028936us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":22442,"thread_start_us":25817,"threads_started":1}
W20260808 11:38:39.152437 7016 heartbeater.cc:644] Failed to heartbeat to 127.24.76.126:44953 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.76.126:44953: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Aug 08 11:38:39 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:39 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:39.161180 7022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:39.153079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.67:42373)
0808 11:38:39.153285 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:39.153304 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:39.153418 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:39.153726 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:39.153731 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:39.154142 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:39.154307 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:39.154316 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:39.155432 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:39.155437 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:39.156386 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:39.156398 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:39.157093 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:39.157099 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:39.157256 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:39.157824 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:39.157842 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:39.160992 (+ 3150us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":70}
W20260808 11:38:39.161367 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:39.153149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:42373 (local address 127.24.76.126:44953)
0808 11:38:39.153327 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:39.153331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:39.153350 (+ 19us) server_negotiation.cc:407] Connection header received
0808 11:38:39.153539 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:39.153544 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:39.153605 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:39.153682 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:39.154444 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:39.155300 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:39.156521 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:39.156927 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:39.157043 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:39.161160 (+ 4117us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.67:42373: BlockingRecv error: recv got EOF from 127.24.76.67:42373 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 08 11:38:39 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:39 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:39.169595 7022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:39.161775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.67:42409)
0808 11:38:39.161920 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:39.161936 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:39.162038 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:39.162377 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:39.162382 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:39.162670 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:39.162813 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:39.162821 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:39.163801 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:39.163804 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:39.165031 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:39.165043 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:39.165795 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:39.165800 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:39.165966 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:39.166422 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:39.166437 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:39.169393 (+ 2956us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 11:38:39.169831 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:39.161868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:42409 (local address 127.24.76.126:44953)
0808 11:38:39.162141 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:39.162147 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:39.162161 (+ 14us) server_negotiation.cc:407] Connection header received
0808 11:38:39.162201 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:39.162204 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:39.162254 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:39.162331 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:39.162944 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:39.163662 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:39.165158 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:39.165670 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:39.165767 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:39.169587 (+ 3820us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.67:42409: BlockingRecv error: recv got EOF from 127.24.76.67:42409 (error 108)
Metrics: {"server-negotiator.queue_time_us":164}
W20260808 11:38:39.169876 7016 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 11:38:39 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:39 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:39.496626 7027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:39.487878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.65:34077)
0808 11:38:39.488347 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:39.488363 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:39.488479 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:39.488772 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:39.488775 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:39.489221 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:39.489377 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:39.489386 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:39.490669 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:39.490674 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:39.491808 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:39.491823 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:39.492493 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:39.492497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:39.492622 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:39.493169 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:39.493186 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:39.496389 (+ 3203us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":256,"threads_started":1}
W20260808 11:38:39.496850 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:39.487953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:34077 (local address 127.24.76.126:44953)
0808 11:38:39.488170 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:39.488176 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:39.488438 (+ 262us) server_negotiation.cc:407] Connection header received
0808 11:38:39.488567 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:39.488571 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:39.488650 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:39.488731 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:39.489538 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:39.490466 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:39.491941 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:39.492352 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:39.492446 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:39.496592 (+ 4146us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.65:34077: BlockingRecv error: recv got EOF from 127.24.76.65:34077 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 08 11:38:39 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:39 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:39.759892 7028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:39.751300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.66:45823)
0808 11:38:39.751784 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:39.751800 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:39.751911 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:39.752178 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:39.752181 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:39.752595 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:39.752742 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:39.752751 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:39.753700 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:39.753703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:39.754695 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:39.754709 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:39.755428 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:39.755433 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:39.755590 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:39.756267 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:39.756286 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:39.759635 (+ 3349us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":272,"threads_started":1}
W20260808 11:38:39.760026 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:39.751351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:45823 (local address 127.24.76.126:44953)
0808 11:38:39.751568 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:39.751576 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:39.751873 (+ 297us) server_negotiation.cc:407] Connection header received
0808 11:38:39.752001 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:39.752004 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:39.752067 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:39.752145 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:39.752845 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:39.753592 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:39.754808 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:39.755231 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:39.755353 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:39.759821 (+ 4468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.66:45823: BlockingRecv error: recv got EOF from 127.24.76.66:45823 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 08 11:38:40 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:40 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:40.179054 7029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:40.170436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.67:38081)
0808 11:38:40.171005 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:40.171025 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:40.171157 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:40.171520 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:40.171525 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:40.172038 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:40.172197 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:40.172206 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:40.173159 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:40.173162 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:40.174161 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:40.174175 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:40.174833 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:40.174837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:40.174950 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:40.175476 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:40.175504 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:40.178808 (+ 3304us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":307,"threads_started":1}
W20260808 11:38:40.179214 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:40.170546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:38081 (local address 127.24.76.126:44953)
0808 11:38:40.170731 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:40.170737 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:40.171101 (+ 364us) server_negotiation.cc:407] Connection header received
0808 11:38:40.171276 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:40.171282 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:40.171376 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:40.171481 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:40.172324 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:40.173064 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:40.174273 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:40.174693 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:40.174780 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:40.179020 (+ 4240us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.67:38081: BlockingRecv error: recv got EOF from 127.24.76.67:38081 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 11:38:40 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:40 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:40.506299 7030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:40.497534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.65:49899)
0808 11:38:40.498004 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:40.498020 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:40.498148 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:40.498470 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:40.498476 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:40.498904 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:40.499105 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:40.499116 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:40.500112 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:40.500118 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:40.501057 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:40.501069 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:40.501757 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:40.501763 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:40.501963 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:40.502522 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:40.502541 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:40.506051 (+ 3510us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":225,"threads_started":1}
W20260808 11:38:40.506521 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:40.497584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:49899 (local address 127.24.76.126:44953)
0808 11:38:40.497790 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:40.497796 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:40.498102 (+ 306us) server_negotiation.cc:407] Connection header received
0808 11:38:40.498261 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:40.498264 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:40.498326 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:40.498407 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:40.499250 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:40.499985 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:40.501197 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:40.501608 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:40.501715 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:40.506248 (+ 4533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.65:49899: BlockingRecv error: recv got EOF from 127.24.76.65:49899 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 11:38:40 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:40 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:40.769193 7032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:40.760770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.66:38765)
0808 11:38:40.761260 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:40.761275 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:40.761406 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:40.761716 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:40.761721 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:40.762171 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:40.762362 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:40.762372 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:40.763353 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:40.763357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:40.764278 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:40.764290 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:40.764955 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:40.764961 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:40.765122 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:40.765646 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:40.765663 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:40.768842 (+ 3179us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":275,"threads_started":1}
W20260808 11:38:40.769273 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:40.760853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:38765 (local address 127.24.76.126:44953)
0808 11:38:40.761055 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:40.761061 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:40.761349 (+ 288us) server_negotiation.cc:407] Connection header received
0808 11:38:40.761514 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:40.761519 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:40.761588 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:40.761668 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:40.762505 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:40.763213 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:40.764412 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:40.764812 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:40.764926 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:40.769034 (+ 4108us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.66:38765: BlockingRecv error: recv got EOF from 127.24.76.66:38765 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 11:38:41 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:41 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:41.188482 7034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:41.179927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.67:55541)
0808 11:38:41.180395 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:41.180411 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:41.180569 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:41.180894 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:41.180898 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:41.181310 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:41.181508 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:41.181517 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:41.182505 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:41.182508 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:41.183406 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:41.183417 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:41.183998 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:41.184002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:41.184121 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:41.184646 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:41.184662 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:41.188156 (+ 3494us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":243,"threads_started":1}
W20260808 11:38:41.188601 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:41.180093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:55541 (local address 127.24.76.126:44953)
0808 11:38:41.180313 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:41.180318 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:41.180477 (+ 159us) server_negotiation.cc:407] Connection header received
0808 11:38:41.180706 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:41.180710 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:41.180781 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:41.180861 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:41.181616 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:41.182397 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:41.183511 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:41.183892 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:41.183975 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:41.188357 (+ 4382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.67:55541: BlockingRecv error: recv got EOF from 127.24.76.67:55541 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 08 11:38:41 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:41 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:41.515878 7038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:41.507211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.65:57033)
0808 11:38:41.507750 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:41.507770 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:41.507893 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:41.508154 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:41.508156 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:41.508502 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:41.508706 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:41.508720 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:41.510105 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:41.510110 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:41.511065 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:41.511079 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:41.511751 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:41.511755 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:41.511877 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:41.512460 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:41.512478 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:41.515643 (+ 3165us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":304,"threads_started":1}
W20260808 11:38:41.516057 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:41.507324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:57033 (local address 127.24.76.126:44953)
0808 11:38:41.507490 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:41.507495 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:41.507831 (+ 336us) server_negotiation.cc:407] Connection header received
0808 11:38:41.507976 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:41.507980 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:41.508041 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:41.508121 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:41.508887 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:41.509960 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:41.511195 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:41.511637 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:41.511731 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:41.515850 (+ 4119us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.65:57033: BlockingRecv error: recv got EOF from 127.24.76.65:57033 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 11:38:41 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:41 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:41.778666 7040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:41.770133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.66:50389)
0808 11:38:41.770632 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:41.770653 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:41.770768 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:41.771029 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:41.771033 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:41.771375 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:41.771554 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:41.771562 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:41.772506 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:41.772510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:41.773423 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:41.773435 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:41.774334 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:41.774340 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:41.774558 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:41.775185 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:41.775202 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:41.778427 (+ 3225us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":253,"threads_started":1}
W20260808 11:38:41.778868 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:41.770212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:50389 (local address 127.24.76.126:44953)
0808 11:38:41.770361 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:41.770366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:41.770714 (+ 348us) server_negotiation.cc:407] Connection header received
0808 11:38:41.770846 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:41.770849 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:41.770908 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:41.770998 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:41.771674 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:41.772404 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:41.773571 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:41.774190 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:41.774324 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:41.778647 (+ 4323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.66:50389: BlockingRecv error: recv got EOF from 127.24.76.66:50389 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 08 11:38:42 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:42 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:42.201335 7041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:42.189458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.67:43451)
0808 11:38:42.189975 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:42.190003 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:42.190129 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:42.190517 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:42.190521 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:42.190994 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:42.191181 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:42.191191 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:42.192324 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:42.192330 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:42.193389 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:42.193403 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:42.194062 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:42.194066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:42.194210 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:42.197752 (+ 3542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:42.197770 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:42.201098 (+ 3328us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":263,"threads_started":1}
W20260808 11:38:42.201475 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:42.189539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:43451 (local address 127.24.76.126:44953)
0808 11:38:42.189740 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:42.189746 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:42.190081 (+ 335us) server_negotiation.cc:407] Connection header received
0808 11:38:42.190246 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:42.190250 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:42.190314 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:42.190423 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:42.191313 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:42.192158 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:42.193513 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:42.193943 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:42.194033 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:42.201289 (+ 7256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.67:43451: BlockingRecv error: recv got EOF from 127.24.76.67:43451 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 11:38:42 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:42 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:42.525296 7042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:42.516780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.65:55001)
0808 11:38:42.517468 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:42.517486 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:42.517597 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:42.517956 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:42.517959 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:42.518351 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:42.518531 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:42.518540 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:42.519520 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:42.519524 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:42.520543 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:42.520555 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:42.521145 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:42.521149 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:42.521262 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:42.521775 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:42.521791 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:42.525041 (+ 3250us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":423,"threads_started":1}
W20260808 11:38:42.525435 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:42.516842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:55001 (local address 127.24.76.126:44953)
0808 11:38:42.517074 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:42.517082 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:42.517558 (+ 476us) server_negotiation.cc:407] Connection header received
0808 11:38:42.517693 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:42.517697 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:42.517759 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:42.517828 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:42.518647 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:42.519384 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:42.520650 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:42.521045 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:42.521130 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:42.525232 (+ 4102us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.65:55001: BlockingRecv error: recv got EOF from 127.24.76.65:55001 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 08 11:38:42 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:42 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:42.787858 7044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:42.779526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.66:41525)
0808 11:38:42.780026 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:42.780045 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:42.780155 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:42.780442 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:42.780445 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:42.780821 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:42.780987 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:42.780997 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:42.781942 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:42.781946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:42.782930 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:42.782941 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:42.783561 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:42.783565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:42.783687 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:42.784250 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:42.784272 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:42.787524 (+ 3252us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":268,"threads_started":1}
W20260808 11:38:42.787959 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:42.779574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:41525 (local address 127.24.76.126:44953)
0808 11:38:42.779722 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:42.779727 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:42.780105 (+ 378us) server_negotiation.cc:407] Connection header received
0808 11:38:42.780258 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:42.780261 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:42.780333 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:42.780403 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:42.781112 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:42.781828 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:42.783043 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:42.783459 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:42.783546 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:42.787727 (+ 4181us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.66:41525: BlockingRecv error: recv got EOF from 127.24.76.66:41525 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 08 11:38:43 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:43 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:43.210842 7045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:43.202266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.67:46439)
0808 11:38:43.202774 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:43.202790 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:43.202921 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:43.203271 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:43.203276 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:43.203671 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:43.203836 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:43.203846 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:43.204848 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:43.204852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:43.205844 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:43.205857 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:43.206545 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:43.206551 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:43.206718 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:43.207230 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:43.207247 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:43.210509 (+ 3262us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":269,"threads_started":1}
W20260808 11:38:43.210983 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:43.202338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:46439 (local address 127.24.76.126:44953)
0808 11:38:43.202540 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:43.202546 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:43.202874 (+ 328us) server_negotiation.cc:407] Connection header received
0808 11:38:43.203060 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:43.203065 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:43.203128 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:43.203211 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:43.203978 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:43.204700 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:43.206009 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:43.206402 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:43.206522 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:43.210715 (+ 4193us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.67:46439: BlockingRecv error: recv got EOF from 127.24.76.67:46439 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 11:38:43 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:43 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:43.536052 7046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:43.526174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.65:51981)
0808 11:38:43.526774 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:43.526798 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:43.526930 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:43.527335 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:43.527340 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:43.527906 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:43.528108 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:43.528122 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:43.529692 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:43.529696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:43.530698 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:43.530711 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:43.531694 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:43.531699 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:43.531851 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:43.532446 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:43.532465 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:43.535809 (+ 3344us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":350,"threads_started":1}
W20260808 11:38:43.536657 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:43.526317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:51981 (local address 127.24.76.126:44953)
0808 11:38:43.526542 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:43.526550 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:43.526879 (+ 329us) server_negotiation.cc:407] Connection header received
0808 11:38:43.527057 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:43.527063 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:43.527146 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:43.527255 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:43.528321 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:43.529534 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:43.530839 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:43.531498 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:43.531633 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:43.536376 (+ 4743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.65:51981: BlockingRecv error: recv got EOF from 127.24.76.65:51981 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 08 11:38:43 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:43 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:43.797844 7048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:43.788848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.66:58615)
0808 11:38:43.789345 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:43.789360 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:43.789489 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:43.789800 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:43.789806 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:43.790279 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:43.790470 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:43.790482 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:43.791480 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:43.791486 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:43.792417 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:43.792430 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:43.793300 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:43.793306 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:43.793471 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:43.794087 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:43.794105 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:43.797619 (+ 3514us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":249,"threads_started":1}
W20260808 11:38:43.798098 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:43.789028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:58615 (local address 127.24.76.126:44953)
0808 11:38:43.789248 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:43.789254 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:43.789433 (+ 179us) server_negotiation.cc:407] Connection header received
0808 11:38:43.789603 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:43.789609 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:43.789669 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:43.789749 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:43.790605 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:43.791343 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:43.792556 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:43.793138 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:43.793249 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:43.797823 (+ 4574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.66:58615: BlockingRecv error: recv got EOF from 127.24.76.66:58615 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 08 11:38:44 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:44 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:44.220141 7050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:44.211885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.67:36035)
0808 11:38:44.212367 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:44.212383 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:44.212510 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:44.212781 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:44.212784 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:44.213197 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:44.213346 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:44.213355 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:44.214311 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:44.214314 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:44.215241 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:44.215253 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:44.215894 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:44.215900 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:44.216059 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:44.216587 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:44.216604 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:44.219823 (+ 3219us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":239,"threads_started":1}
W20260808 11:38:44.220302 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:44.211920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:36035 (local address 127.24.76.126:44953)
0808 11:38:44.212116 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:44.212123 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:44.212458 (+ 335us) server_negotiation.cc:407] Connection header received
0808 11:38:44.212605 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:44.212609 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:44.212670 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:44.212748 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:44.213454 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:44.214212 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:44.215353 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:44.215744 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:44.215849 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:44.220054 (+ 4205us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.67:36035: BlockingRecv error: recv got EOF from 127.24.76.67:36035 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 11:38:44 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:44 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:44.545538 7052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:44.536898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.65:57885)
0808 11:38:44.537408 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:44.537426 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:44.537543 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:44.537801 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:44.537804 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:44.538245 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:44.538419 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:44.538429 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:44.539330 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:44.539333 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:44.540304 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:44.540318 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:44.540922 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:44.540926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:44.541061 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:44.541647 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:44.541672 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:44.545224 (+ 3552us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":271,"threads_started":1}
W20260808 11:38:44.545691 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:44.536995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:57885 (local address 127.24.76.126:44953)
0808 11:38:44.537168 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:44.537173 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:44.537488 (+ 315us) server_negotiation.cc:407] Connection header received
0808 11:38:44.537630 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:44.537634 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:44.537692 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:44.537770 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:44.538526 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:44.539219 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:44.540417 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:44.540808 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:44.540893 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:44.545437 (+ 4544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.65:57885: BlockingRecv error: recv got EOF from 127.24.76.65:57885 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 11:38:44 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:44 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:44.807137 7053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:44.798906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.66:48967)
0808 11:38:44.799373 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:44.799393 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:44.799514 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:44.799792 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:44.799795 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:44.800216 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:44.800411 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:44.800421 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:44.801346 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:44.801350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:44.802313 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:44.802325 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:44.802961 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:44.802965 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:44.803097 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:44.803641 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:44.803658 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:44.806901 (+ 3243us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":243,"threads_started":1}
W20260808 11:38:44.807266 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:44.798973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:48967 (local address 127.24.76.126:44953)
0808 11:38:44.799142 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:44.799147 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:44.799477 (+ 330us) server_negotiation.cc:407] Connection header received
0808 11:38:44.799608 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:44.799612 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:44.799688 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:44.799761 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:44.800527 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:44.801225 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:44.802417 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:44.802847 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:44.802936 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:44.807091 (+ 4155us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.66:48967: BlockingRecv error: recv got EOF from 127.24.76.66:48967 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 11:38:45 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:45 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:45.228976 7054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:45.221040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.67:52947)
0808 11:38:45.221503 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:45.221519 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:45.221632 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:45.221910 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:45.221913 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:45.222313 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:45.222479 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:45.222488 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:45.223449 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:45.223452 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:45.224364 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:45.224375 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:45.224996 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:45.224999 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:45.225121 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:45.225663 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:45.225683 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:45.228728 (+ 3045us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":236,"threads_started":1}
W20260808 11:38:45.229130 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:45.221123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:52947 (local address 127.24.76.126:44953)
0808 11:38:45.221290 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:45.221295 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:45.221583 (+ 288us) server_negotiation.cc:407] Connection header received
0808 11:38:45.221739 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:45.221743 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:45.221805 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:45.221875 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:45.222592 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:45.223344 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:45.224466 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:45.224872 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:45.224959 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:45.228915 (+ 3956us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.67:52947: BlockingRecv error: recv got EOF from 127.24.76.67:52947 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 08 11:38:45 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:45 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:45.555640 7055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:45.546548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.65:40213)
0808 11:38:45.547108 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:45.547125 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:45.547237 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:45.547496 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:45.547499 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:45.547893 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:45.548091 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:45.548100 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:45.549010 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:45.549013 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:45.549973 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:45.549986 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:45.550652 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:45.550656 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:45.550776 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:45.551534 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:45.551558 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:45.555392 (+ 3834us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":253,"threads_started":1}
W20260808 11:38:45.555717 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:45.546603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:40213 (local address 127.24.76.126:44953)
0808 11:38:45.546784 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:45.546788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:45.547187 (+ 399us) server_negotiation.cc:407] Connection header received
0808 11:38:45.547324 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:45.547328 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:45.547388 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:45.547467 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:45.548204 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:45.548908 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:45.550090 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:45.550518 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:45.550606 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:45.555534 (+ 4928us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.65:40213: BlockingRecv error: recv got EOF from 127.24.76.65:40213 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 11:38:45 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:45 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:45.817138 7056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:45.808336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.66:42613)
0808 11:38:45.809104 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:45.809120 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:45.809236 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:45.809606 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:45.809612 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:45.810161 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:45.810338 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:45.810348 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:45.811385 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:45.811388 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:45.812321 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:45.812332 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:45.812983 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:45.812988 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:45.813100 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:45.813656 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:45.813672 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:45.816860 (+ 3188us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":660,"thread_start_us":439,"threads_started":1}
W20260808 11:38:45.817322 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:45.808420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:42613 (local address 127.24.76.126:44953)
0808 11:38:45.808646 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:45.808653 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:45.809190 (+ 537us) server_negotiation.cc:407] Connection header received
0808 11:38:45.809366 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:45.809372 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:45.809448 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:45.809551 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:45.810497 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:45.811271 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:45.812440 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:45.812870 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:45.812961 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:45.817090 (+ 4129us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.66:42613: BlockingRecv error: recv got EOF from 127.24.76.66:42613 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Aug 08 11:38:46 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:46 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:46.238394 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:46.229846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.67:43117)
0808 11:38:46.230426 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:46.230443 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:46.230563 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:46.230832 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:46.230835 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:46.231235 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:46.231411 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:46.231420 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:46.232333 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:46.232336 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:46.233276 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:46.233289 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:46.233948 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:46.233954 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:46.234101 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:46.234724 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:46.234742 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:46.238137 (+ 3395us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":241,"threads_started":1}
W20260808 11:38:46.238497 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:46.229885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:43117 (local address 127.24.76.126:44953)
0808 11:38:46.230105 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:46.230111 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:46.230520 (+ 409us) server_negotiation.cc:407] Connection header received
0808 11:38:46.230658 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:46.230662 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:46.230725 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:46.230803 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:46.231519 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:46.232223 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:46.233388 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:46.233814 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:46.233932 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:46.238310 (+ 4378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.67:43117: BlockingRecv error: recv got EOF from 127.24.76.67:43117 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 08 11:38:46 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:46 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:46.565443 7062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:46.556548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.65:39907)
0808 11:38:46.557095 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:46.557112 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:46.557230 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:46.557496 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:46.557500 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:46.557948 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:46.558128 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:46.558138 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:46.559199 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:46.559203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:46.560150 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:46.560163 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:46.560872 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:46.560877 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:46.560991 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:46.561564 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:46.561582 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:46.565150 (+ 3568us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":279,"threads_started":1}
W20260808 11:38:46.565546 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:46.556763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:39907 (local address 127.24.76.126:44953)
0808 11:38:46.556980 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:46.556986 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:46.557181 (+ 195us) server_negotiation.cc:407] Connection header received
0808 11:38:46.557318 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:46.557322 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:46.557384 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:46.557472 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:46.558266 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:46.559059 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:46.560294 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:46.560719 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:46.560811 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:46.565320 (+ 4509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.65:39907: BlockingRecv error: recv got EOF from 127.24.76.65:39907 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 08 11:38:46 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:46 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:46.826818 7063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:46.818027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.66:47285)
0808 11:38:46.818506 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:46.818522 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:46.818653 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:46.818979 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:46.818984 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:46.819396 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:46.819570 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:46.819579 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:46.820745 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:46.820749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:46.821690 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:46.821702 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:46.822523 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:46.822530 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:46.822682 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:46.823225 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:46.823242 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:46.826565 (+ 3323us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":229,"threads_started":1}
W20260808 11:38:46.827045 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:46.818066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:47285 (local address 127.24.76.126:44953)
0808 11:38:46.818265 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:46.818272 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:46.818604 (+ 332us) server_negotiation.cc:407] Connection header received
0808 11:38:46.818765 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:46.818770 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:46.818840 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:46.818923 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:46.819714 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:46.820597 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:46.821832 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:46.822337 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:46.822457 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:46.826814 (+ 4357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.66:47285: BlockingRecv error: recv got EOF from 127.24.76.66:47285 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 08 11:38:47 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:47 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:47.250350 7065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:47.239214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.67:47519)
0808 11:38:47.239699 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:47.239715 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:47.239825 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:47.240105 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:47.240108 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:47.240524 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:47.240710 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:47.240719 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:47.241759 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:47.241765 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:47.243095 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:47.243108 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:47.243761 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:47.243765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:47.243872 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:47.247060 (+ 3188us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:47.247078 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:47.250111 (+ 3033us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":259,"threads_started":1}
W20260808 11:38:47.250502 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:47.239288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:47519 (local address 127.24.76.126:44953)
0808 11:38:47.239508 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:47.239514 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:47.239782 (+ 268us) server_negotiation.cc:407] Connection header received
0808 11:38:47.239912 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:47.239916 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:47.239980 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:47.240070 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:47.240811 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:47.241607 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:47.243214 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:47.243624 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:47.243713 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:47.250289 (+ 6576us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.67:47519: BlockingRecv error: recv got EOF from 127.24.76.67:47519 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 08 11:38:47 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:47 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:47.574898 7068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:47.566396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.65:55679)
0808 11:38:47.566902 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:47.566918 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:47.567033 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:47.567294 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:47.567297 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:47.567718 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:47.567874 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:47.567884 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:47.569004 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:47.569008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:47.569976 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:47.569989 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:47.570752 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:47.570756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:47.570870 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:47.571378 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:47.571394 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:47.574677 (+ 3283us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":270,"threads_started":1}
W20260808 11:38:47.575050 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:47.566429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:55679 (local address 127.24.76.126:44953)
0808 11:38:47.566600 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:47.566604 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:47.567001 (+ 397us) server_negotiation.cc:407] Connection header received
0808 11:38:47.567120 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:47.567124 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:47.567200 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:47.567280 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:47.568048 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:47.568873 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:47.570101 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:47.570638 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:47.570737 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:47.574828 (+ 4091us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.65:55679: BlockingRecv error: recv got EOF from 127.24.76.65:55679 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 11:38:47 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:47 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:47.836477 7070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:47.827878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.66:36279)
0808 11:38:47.828383 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:47.828400 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:47.828520 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:47.828781 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:47.828783 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:47.829180 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:47.829334 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:47.829343 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:47.830477 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:47.830489 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:47.831428 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:47.831442 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:47.832165 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:47.832171 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:47.832331 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:47.832865 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:47.832881 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:47.836248 (+ 3367us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":239,"threads_started":1}
W20260808 11:38:47.836663 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:47.827977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:36279 (local address 127.24.76.126:44953)
0808 11:38:47.828190 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:47.828196 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:47.828469 (+ 273us) server_negotiation.cc:407] Connection header received
0808 11:38:47.828625 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:47.828629 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:47.828694 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:47.828763 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:47.829443 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:47.830337 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:47.831576 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:47.832013 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:47.832120 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:47.836423 (+ 4303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.66:36279: BlockingRecv error: recv got EOF from 127.24.76.66:36279 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 08 11:38:48 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:48 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:48.259706 7071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:48.251230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.67:45045)
0808 11:38:48.251723 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:48.251739 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:48.251862 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:48.252192 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:48.252197 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:48.252624 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:48.252771 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:48.252781 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:48.253767 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:48.253771 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:48.254744 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:48.254756 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:48.255581 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:48.255587 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:48.255758 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:48.256312 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:48.256330 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:48.259394 (+ 3064us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":257,"threads_started":1}
W20260808 11:38:48.259824 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:48.251294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:45045 (local address 127.24.76.126:44953)
0808 11:38:48.251488 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:48.251494 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:48.251818 (+ 324us) server_negotiation.cc:407] Connection header received
0808 11:38:48.251995 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:48.251999 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:48.252061 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:48.252145 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:48.252912 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:48.253623 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:48.254899 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:48.255432 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:48.255543 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:48.259588 (+ 4045us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.67:45045: BlockingRecv error: recv got EOF from 127.24.76.67:45045 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 11:38:48 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:48 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:48.584545 7073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:48.575820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.65:44859)
0808 11:38:48.576318 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:48.576334 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:48.576460 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:48.576739 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:48.576742 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:48.577118 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:48.577284 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:48.577292 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:48.578536 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:48.578540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:48.579676 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:48.579692 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:48.580339 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:48.580343 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:48.580471 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:48.581067 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:48.581085 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:48.584274 (+ 3189us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":271,"threads_started":1}
W20260808 11:38:48.584659 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:48.575872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:44859 (local address 127.24.76.126:44953)
0808 11:38:48.576031 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:48.576036 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:48.576400 (+ 364us) server_negotiation.cc:407] Connection header received
0808 11:38:48.576556 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:48.576560 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:48.576622 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:48.576702 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:48.577411 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:48.578406 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:48.579795 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:48.580211 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:48.580321 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:48.584445 (+ 4124us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.65:44859: BlockingRecv error: recv got EOF from 127.24.76.65:44859 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 11:38:48 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:48 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:48.845666 7075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:48.837304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.66:38641)
0808 11:38:48.837801 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:48.837817 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:48.837968 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:48.838245 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:48.838248 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:48.838631 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:48.838799 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:48.838808 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:48.839723 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:48.839726 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:48.840734 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:48.840747 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:48.841369 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:48.841373 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:48.841489 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:48.842079 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:48.842097 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:48.845363 (+ 3266us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":275,"threads_started":1}
W20260808 11:38:48.845813 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:48.837378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:38641 (local address 127.24.76.126:44953)
0808 11:38:48.837534 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:48.837538 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:48.837877 (+ 339us) server_negotiation.cc:407] Connection header received
0808 11:38:48.838067 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:48.838070 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:48.838134 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:48.838214 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:48.838912 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:48.839614 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:48.840842 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:48.841251 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:48.841338 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:48.845572 (+ 4234us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.66:38641: BlockingRecv error: recv got EOF from 127.24.76.66:38641 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 11:38:49 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:49 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:49.269307 7077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:49.260620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.67:42895)
0808 11:38:49.261104 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:49.261122 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:49.261232 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:49.261613 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:49.261617 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:49.262074 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:49.262251 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:49.262261 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:49.263326 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:49.263330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:49.264315 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:49.264327 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:49.264951 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:49.264954 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:49.265059 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:49.265651 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:49.265670 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:49.268983 (+ 3313us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":246,"threads_started":1}
W20260808 11:38:49.269467 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:49.260767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:42895 (local address 127.24.76.126:44953)
0808 11:38:49.260996 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:49.261001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:49.261186 (+ 185us) server_negotiation.cc:407] Connection header received
0808 11:38:49.261329 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:49.261333 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:49.261409 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:49.261484 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:49.262400 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:49.263158 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:49.264420 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:49.264842 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:49.264929 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:49.269207 (+ 4278us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.67:42895: BlockingRecv error: recv got EOF from 127.24.76.67:42895 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 08 11:38:49 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:49 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:49.594278 7079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:49.585464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.65:34089)
0808 11:38:49.585959 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:49.585978 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:49.586091 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:49.586367 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:49.586370 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:49.586791 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:49.586980 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:49.586990 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:49.587941 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:49.587944 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:49.588851 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:49.588862 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:49.589563 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:49.589567 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:49.589687 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:49.590334 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:49.590353 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:49.594052 (+ 3699us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":247,"threads_started":1}
W20260808 11:38:49.594406 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:49.585510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:34089 (local address 127.24.76.126:44953)
0808 11:38:49.585721 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:49.585728 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:49.586048 (+ 320us) server_negotiation.cc:407] Connection header received
0808 11:38:49.586176 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:49.586179 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:49.586255 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:49.586333 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:49.587094 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:49.587816 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:49.588991 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:49.589441 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:49.589536 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:49.594199 (+ 4663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.65:34089: BlockingRecv error: recv got EOF from 127.24.76.65:34089 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 08 11:38:49 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:49 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:49.855432 7080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:49.846749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.66:58349)
0808 11:38:49.847240 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:49.847260 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:49.847379 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:49.847633 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:49.847636 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:49.848075 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:49.848312 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:49.848323 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:49.849554 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:49.849559 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:49.850518 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:49.850533 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:49.851159 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:49.851163 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:49.851286 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:49.851892 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:49.851909 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:49.855213 (+ 3304us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":265,"threads_started":1}
W20260808 11:38:49.855608 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:49.846820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:58349 (local address 127.24.76.126:44953)
0808 11:38:49.846988 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:49.846992 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:49.847324 (+ 332us) server_negotiation.cc:407] Connection header received
0808 11:38:49.847462 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:49.847466 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:49.847524 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:49.847603 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:49.848463 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:49.849431 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:49.850643 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:49.851039 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:49.851128 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:49.855380 (+ 4252us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.66:58349: BlockingRecv error: recv got EOF from 127.24.76.66:58349 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 11:38:50 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:50 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:50.279035 7082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:50.270239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.67:49987)
0808 11:38:50.270709 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:50.270724 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:50.270853 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:50.271180 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:50.271185 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:50.271617 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:50.271817 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:50.271826 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:50.272825 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:50.272829 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:50.273734 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:50.273746 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:50.274464 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:50.274470 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:50.274633 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:50.275164 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:50.275181 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:50.278813 (+ 3632us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":254,"threads_started":1}
W20260808 11:38:50.279161 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:50.270331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:49987 (local address 127.24.76.126:44953)
0808 11:38:50.270530 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:50.270536 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:50.270805 (+ 269us) server_negotiation.cc:407] Connection header received
0808 11:38:50.270972 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:50.270976 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:50.271044 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:50.271126 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:50.271961 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:50.272693 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:50.273881 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:50.274313 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:50.274425 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:50.278974 (+ 4549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.67:49987: BlockingRecv error: recv got EOF from 127.24.76.67:49987 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 11:38:50 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:50 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:50.604050 7083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:50.595190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.65:60767)
0808 11:38:50.595742 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:50.595759 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:50.595869 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:50.596202 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:50.596205 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:50.596641 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:50.596818 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:50.596828 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:50.597826 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:50.597830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:50.598807 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:50.598821 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:50.599558 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:50.599563 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:50.599700 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:50.600356 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:50.600380 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:50.603792 (+ 3412us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":307,"threads_started":1}
W20260808 11:38:50.604189 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:50.595248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:60767 (local address 127.24.76.126:44953)
0808 11:38:50.595453 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:50.595459 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:50.595819 (+ 360us) server_negotiation.cc:407] Connection header received
0808 11:38:50.595963 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:50.595967 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:50.596079 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:50.596179 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:50.596944 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:50.597717 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:50.598936 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:50.599404 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:50.599518 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:50.603965 (+ 4447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.65:60767: BlockingRecv error: recv got EOF from 127.24.76.65:60767 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 11:38:50 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:50 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:50.864586 7085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:50.856282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.66:49371)
0808 11:38:50.856866 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:50.856882 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:50.857016 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:50.857293 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:50.857296 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:50.857721 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:50.857931 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:50.857943 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:50.858927 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:50.858931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:50.859855 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:50.859868 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:50.860521 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:50.860525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:50.860646 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:50.861212 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:50.861230 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:50.864347 (+ 3117us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":323,"threads_started":1}
W20260808 11:38:50.864750 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:50.856341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:49371 (local address 127.24.76.126:44953)
0808 11:38:50.856516 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:50.856521 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:50.856956 (+ 435us) server_negotiation.cc:407] Connection header received
0808 11:38:50.857108 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:50.857112 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:50.857183 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:50.857262 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:50.858064 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:50.858803 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:50.859962 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:50.860382 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:50.860479 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:50.864526 (+ 4047us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.66:49371: BlockingRecv error: recv got EOF from 127.24.76.66:49371 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 11:38:51 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:51 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:51.288764 7086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:51.279936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.67:46841)
0808 11:38:51.280495 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:51.280511 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:51.280654 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:51.280991 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:51.280996 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:51.281519 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:51.281743 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:51.281754 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:51.282829 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:51.282835 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:51.283769 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:51.283781 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:51.284490 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:51.284497 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:51.284688 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:51.285268 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:51.285286 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:51.288513 (+ 3227us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":260,"threads_started":1}
W20260808 11:38:51.288939 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:51.279986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:46841 (local address 127.24.76.126:44953)
0808 11:38:51.280176 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:51.280183 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:51.280600 (+ 417us) server_negotiation.cc:407] Connection header received
0808 11:38:51.280778 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:51.280783 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:51.280846 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:51.280926 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:51.281880 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:51.282693 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:51.283908 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:51.284307 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:51.284423 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:51.288715 (+ 4292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.67:46841: BlockingRecv error: recv got EOF from 127.24.76.67:46841 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 08 11:38:51 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:51 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:51.613363 7087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:51.605021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.65:54211)
0808 11:38:51.605511 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:51.605531 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:51.605668 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:51.606084 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:51.606090 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:51.606526 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:51.606681 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:51.606690 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:51.607632 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:51.607636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:51.608571 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:51.608582 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:51.609206 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:51.609210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:51.609334 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:51.609863 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:51.609880 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:51.613134 (+ 3254us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":246,"threads_started":1}
W20260808 11:38:51.613520 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:51.605061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:54211 (local address 127.24.76.126:44953)
0808 11:38:51.605244 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:51.605251 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:51.605618 (+ 367us) server_negotiation.cc:407] Connection header received
0808 11:38:51.605785 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:51.605791 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:51.605919 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:51.606029 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:51.606813 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:51.607510 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:51.608686 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:51.609099 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:51.609186 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:51.613321 (+ 4135us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.65:54211: BlockingRecv error: recv got EOF from 127.24.76.65:54211 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 11:38:51 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:51 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:51.873796 7088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:51.865459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.66:53029)
0808 11:38:51.865911 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:51.865928 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:51.866046 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:51.866365 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:51.866368 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:51.866779 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:51.866934 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:51.866942 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:51.867860 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:51.867864 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:51.868757 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:51.868772 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:51.869553 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:51.869557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:51.869683 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:51.870323 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:51.870345 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:51.873567 (+ 3222us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":225,"threads_started":1}
W20260808 11:38:51.873991 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:51.865605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:53029 (local address 127.24.76.126:44953)
0808 11:38:51.865842 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:51.865849 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:51.865996 (+ 147us) server_negotiation.cc:407] Connection header received
0808 11:38:51.866177 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:51.866183 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:51.866258 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:51.866333 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:51.867039 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:51.867754 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:51.868895 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:51.869432 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:51.869527 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:51.873751 (+ 4224us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.66:53029: BlockingRecv error: recv got EOF from 127.24.76.66:53029 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 08 11:38:52 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:52 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.67@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:52.301695 7089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:52.289652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.67:38515)
0808 11:38:52.290159 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:52.290174 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:52.290304 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:52.290630 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:52.290634 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:52.291077 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:52.291253 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:52.291263 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:52.292263 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:52.292269 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:52.293201 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:52.293214 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:52.293933 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:52.293939 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:52.294111 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:52.298176 (+ 4065us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:52.298194 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:52.301387 (+ 3193us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":280,"threads_started":1}
W20260808 11:38:52.301831 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:52.289720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:38515 (local address 127.24.76.126:44953)
0808 11:38:52.289940 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:52.289946 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:52.290253 (+ 307us) server_negotiation.cc:407] Connection header received
0808 11:38:52.290425 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:52.290428 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:52.290497 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:52.290576 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:52.291389 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:52.292122 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:52.293347 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:52.293772 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:52.293878 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:52.301614 (+ 7736us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.67:38515: BlockingRecv error: recv got EOF from 127.24.76.67:38515 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 11:38:52 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:52 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.65@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:52.622583 7090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:52.614451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.65:51007)
0808 11:38:52.614901 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:52.614917 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:52.615052 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:52.615311 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:52.615314 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:52.615683 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:52.615857 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:52.615865 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:52.616822 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:52.616826 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:52.617750 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:52.617761 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:52.618394 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:52.618397 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:52.618528 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:52.619099 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:52.619117 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:52.622335 (+ 3218us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":212,"threads_started":1}
W20260808 11:38:52.622746 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:52.614539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:51007 (local address 127.24.76.126:44953)
0808 11:38:52.614876 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:52.614883 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:52.615009 (+ 126us) server_negotiation.cc:407] Connection header received
0808 11:38:52.615131 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:52.615134 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:52.615207 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:52.615285 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:52.616001 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:52.616709 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:52.617854 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:52.618287 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:52.618374 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:52.622504 (+ 4130us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.65:51007: BlockingRecv error: recv got EOF from 127.24.76.65:51007 (error 108)
Metrics: {"server-negotiator.queue_time_us":195}
Aug 08 11:38:52 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 11:38:52 dist-test-slave-tc2s krb5kdc[5451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.76.66@KRBTEST.COM for oryx/127.24.76.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 11:38:52.883698 7091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:52.874892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:44953 (local address 127.24.76.66:43389)
0808 11:38:52.875368 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:52.875389 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:52.875531 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:52.875922 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:52.875927 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:52.876449 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:52.876681 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:52.876695 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:52.877680 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:52.877683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:52.878795 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:52.878809 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:52.879430 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:52.879434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:52.879567 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:52.880168 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:38:52.880186 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:52.883447 (+ 3261us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.76.126:44953: Server oryx/127.24.76.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":215,"threads_started":1}
W20260808 11:38:52.883819 7023 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:52.874972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:43389 (local address 127.24.76.126:44953)
0808 11:38:52.875214 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:52.875219 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:52.875477 (+ 258us) server_negotiation.cc:407] Connection header received
0808 11:38:52.875682 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:52.875687 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:52.875777 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:52.875879 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:52.876827 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:52.877551 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:52.878910 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:52.879321 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:52.879405 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:52.883636 (+ 4231us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.76.66:43389: BlockingRecv error: recv got EOF from 127.24.76.66:43389 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 11:38:53.070849 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 6364
I20260808 11:38:53.082791 7094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:53.071872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56372 (local address 127.24.76.65:36707)
0808 11:38:53.072561 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:53.072568 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:53.072597 (+ 29us) server_negotiation.cc:407] Connection header received
0808 11:38:53.072658 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:53.072665 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:53.072887 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:53.073060 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:53.074450 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:53.075590 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:53.076748 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:53.077299 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:53.077396 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:53.079809 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:53.079818 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:53.079833 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:53.079872 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:53.082285 (+ 2413us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.24.76.65@KRBTEST.COM not found in keytab (ticket kvno 2)
0808 11:38:53.082553 (+ 268us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56372: Request ticket server kudu/127.24.76.65@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":357,"threads_started":1}
W20260808 11:38:53.082883 7094 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56372: Request ticket server kudu/127.24.76.65@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260808 11:38:53.082880 24881 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 6364: Not authorized: Client connection negotiation failed: client connection to 127.24.76.65:36707: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.24.76.65@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260808 11:38:53.083091 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 6364
I20260808 11:38:53.107383 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 6605
I20260808 11:38:53.119738 7095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:53.108447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49216 (local address 127.24.76.66:37737)
0808 11:38:53.109159 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:53.109166 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:53.109197 (+ 31us) server_negotiation.cc:407] Connection header received
0808 11:38:53.109262 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:53.109268 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:53.109499 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:53.109688 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:53.111037 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:53.112108 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:53.113332 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:53.114033 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:53.114154 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:53.116582 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:53.116591 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:53.116604 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:53.116644 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:53.119088 (+ 2444us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.24.76.66@KRBTEST.COM not found in keytab (ticket kvno 2)
0808 11:38:53.119408 (+ 320us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49216: Request ticket server kudu/127.24.76.66@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":274,"threads_started":1}
W20260808 11:38:53.119771 24881 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 6605: Not authorized: Client connection negotiation failed: client connection to 127.24.76.66:37737: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.24.76.66@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260808 11:38:53.119851 7095 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49216: Request ticket server kudu/127.24.76.66@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260808 11:38:53.119904 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 6605
I20260808 11:38:53.139791 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 6819
I20260808 11:38:53.151504 7096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:53.140870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47624 (local address 127.24.76.67:46685)
0808 11:38:53.141386 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:53.141393 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:53.141418 (+ 25us) server_negotiation.cc:407] Connection header received
0808 11:38:53.141485 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:53.141500 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:53.141731 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:53.141938 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:53.143242 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:53.144226 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:53.145433 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:53.146055 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:53.146169 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:53.148550 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:53.148559 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:53.148571 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:38:53.148610 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:53.150874 (+ 2264us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.24.76.67@KRBTEST.COM not found in keytab (ticket kvno 2)
0808 11:38:53.151175 (+ 301us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47624: Request ticket server kudu/127.24.76.67@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":235,"threads_started":1}
W20260808 11:38:53.151527 24881 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 6819: Not authorized: Client connection negotiation failed: client connection to 127.24.76.67:46685: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.24.76.67@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260808 11:38:53.151597 7096 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47624: Request ticket server kudu/127.24.76.67@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260808 11:38:53.151674 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 6819
I20260808 11:38:53.177016 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 6256
I20260808 11:38:53.178368 6353 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:53.297348 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 6256
2026-08-08T11:38:53Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21273 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T11:38:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:38:53Z Disabled control of system clock
I20260808 11:38:53.388875 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:37811
--webserver_interface=127.24.76.126
--webserver_port=0
--builtin_ntp_servers=127.24.76.84:38279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:37811
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 11:38:53.712896 7105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:53.713176 7105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:53.713295 7105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:53.720254 7105 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:38:53.720387 7105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:53.720454 7105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:53.720541 7105 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:38:53.720597 7105 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:38:53.729435 7105 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:38279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:37811
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:37811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:53.730686 7105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:53.732396 7105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:53.743410 7122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:53.743462 7118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:53.743924 7116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:53.744244 7105 server_base.cc:1034] running on GCE node
I20260808 11:38:53.744998 7105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:53.746300 7105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:53.747496 7105 hybrid_clock.cc:648] HybridClock initialized: now 1786189133747448 us; error 42 us; skew 500 ppm
I20260808 11:38:53.747823 7105 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:38:53.749987 7105 webserver.cc:460] Webserver started at http://127.24.76.126:33261/ using document root <none> and password file <none>
I20260808 11:38:53.750680 7105 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:53.750773 7105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:53.751087 7105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:53.753966 7105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "2451a4315a0647e0b65c5fbf30118319"
format_stamp: "Formatted at 2026-08-08 11:38:53 on dist-test-slave-tc2s"
server_key: "580b61257074703c8579fdce81308b74"
server_key_iv: "821efd66df95bb84ae34422cf34ec3a6"
server_key_version: "encryptionkey@0"
I20260808 11:38:53.754734 7105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "2451a4315a0647e0b65c5fbf30118319"
format_stamp: "Formatted at 2026-08-08 11:38:53 on dist-test-slave-tc2s"
server_key: "580b61257074703c8579fdce81308b74"
server_key_iv: "821efd66df95bb84ae34422cf34ec3a6"
server_key_version: "encryptionkey@0"
I20260808 11:38:53.759581 7105 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 11:38:53.762928 7132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:53.764243 7105 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 11:38:53.764412 7105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "2451a4315a0647e0b65c5fbf30118319"
format_stamp: "Formatted at 2026-08-08 11:38:53 on dist-test-slave-tc2s"
server_key: "580b61257074703c8579fdce81308b74"
server_key_iv: "821efd66df95bb84ae34422cf34ec3a6"
server_key_version: "encryptionkey@0"
I20260808 11:38:53.764604 7105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:53.792680 7105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:53.793480 7105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:53.793744 7105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:53.811020 7105 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:37811
I20260808 11:38:53.811064 7208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:37811 every 8 connection(s)
I20260808 11:38:53.812278 7105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:38:53.815901 7209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:53.820803 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 7105
I20260808 11:38:53.821008 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:38:53.821861 24881 external_mini_cluster.cc:1428] Setting key 72214b0f5a5e5a16af53d7e4ab1aa15e
I20260808 11:38:53.827378 7209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2451a4315a0647e0b65c5fbf30118319: Bootstrap starting.
I20260808 11:38:53.830427 7209 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2451a4315a0647e0b65c5fbf30118319: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:53.831427 7209 log.cc:824] T 00000000000000000000000000000000 P 2451a4315a0647e0b65c5fbf30118319: Log is configured to *not* fsync() on all Append() calls
I20260808 11:38:53.834760 7209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2451a4315a0647e0b65c5fbf30118319: No bootstrap required, opened a new log
I20260808 11:38:53.836051 7214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:53.824145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41432 (local address 127.24.76.126:37811)
0808 11:38:53.825257 (+ 1112us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:53.825267 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:53.825319 (+ 52us) server_negotiation.cc:407] Connection header received
0808 11:38:53.827005 (+ 1686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:53.827029 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:53.827476 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:53.827845 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:53.828485 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:53.829808 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:53.831237 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:53.832028 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:53.832216 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:53.833279 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:53.833290 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:53.833308 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:38:53.834026 (+ 718us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:53.834135 (+ 109us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:38:53.834162 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:53.834280 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:53.834475 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 11:38:53.834800 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":413,"threads_started":1}
I20260808 11:38:53.842983 7209 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2451a4315a0647e0b65c5fbf30118319 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2451a4315a0647e0b65c5fbf30118319" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 37811 } }
I20260808 11:38:53.843320 7209 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2451a4315a0647e0b65c5fbf30118319 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:53.843405 7209 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2451a4315a0647e0b65c5fbf30118319 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2451a4315a0647e0b65c5fbf30118319, State: Initialized, Role: FOLLOWER
I20260808 11:38:53.844008 7209 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2451a4315a0647e0b65c5fbf30118319 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2451a4315a0647e0b65c5fbf30118319" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 37811 } }
I20260808 11:38:53.844291 7209 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2451a4315a0647e0b65c5fbf30118319 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:38:53.844440 7209 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2451a4315a0647e0b65c5fbf30118319 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:38:53.844592 7209 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2451a4315a0647e0b65c5fbf30118319 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:53.846813 7209 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2451a4315a0647e0b65c5fbf30118319 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2451a4315a0647e0b65c5fbf30118319" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 37811 } }
I20260808 11:38:53.847272 7209 leader_election.cc:302] T 00000000000000000000000000000000 P 2451a4315a0647e0b65c5fbf30118319 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2451a4315a0647e0b65c5fbf30118319; no voters:
I20260808 11:38:53.847841 7209 leader_election.cc:288] T 00000000000000000000000000000000 P 2451a4315a0647e0b65c5fbf30118319 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:38:53.848006 7217 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2451a4315a0647e0b65c5fbf30118319 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:38:53.848693 7217 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2451a4315a0647e0b65c5fbf30118319 [term 1 LEADER]: Becoming Leader. State: Replica: 2451a4315a0647e0b65c5fbf30118319, State: Running, Role: LEADER
I20260808 11:38:53.849231 7217 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2451a4315a0647e0b65c5fbf30118319 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2451a4315a0647e0b65c5fbf30118319" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 37811 } }
I20260808 11:38:53.849823 7209 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2451a4315a0647e0b65c5fbf30118319 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:38:53.852389 7218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2451a4315a0647e0b65c5fbf30118319 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2451a4315a0647e0b65c5fbf30118319" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2451a4315a0647e0b65c5fbf30118319" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 37811 } } }
I20260808 11:38:53.852398 7219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2451a4315a0647e0b65c5fbf30118319 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2451a4315a0647e0b65c5fbf30118319. Latest consensus state: current_term: 1 leader_uuid: "2451a4315a0647e0b65c5fbf30118319" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2451a4315a0647e0b65c5fbf30118319" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 37811 } } }
I20260808 11:38:53.853089 7219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2451a4315a0647e0b65c5fbf30118319 [sys.catalog]: This master's current role is: LEADER
I20260808 11:38:53.853085 7218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2451a4315a0647e0b65c5fbf30118319 [sys.catalog]: This master's current role is: LEADER
I20260808 11:38:53.854506 7228 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:38:53.859347 7228 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:38:53.867656 7228 catalog_manager.cc:1347] Generated new cluster ID: 95326e7bc7544d1ca0d8795b3eac78ef
I20260808 11:38:53.867785 7228 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:38:53.897128 7228 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:38:53.898485 7228 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:38:53.908357 7228 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2451a4315a0647e0b65c5fbf30118319: Generated new TSK 0
I20260808 11:38:53.909109 7228 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:38:53.919880 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.65:0
--local_ip_for_outbound_sockets=127.24.76.65
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:37811
--builtin_ntp_servers=127.24.76.84:38279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 11:38:54.273025 7250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:54.273278 7250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:54.273396 7250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:54.280736 7250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:54.280834 7250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:54.280974 7250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.65
I20260808 11:38:54.289799 7250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:38279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:37811
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.65
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:54.291054 7250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:54.292789 7250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:54.305335 7259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:54.305466 7263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:54.305526 7260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:54.306885 7250 server_base.cc:1034] running on GCE node
I20260808 11:38:54.307462 7250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:54.308343 7250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:54.309543 7250 hybrid_clock.cc:648] HybridClock initialized: now 1786189134309496 us; error 47 us; skew 500 ppm
I20260808 11:38:54.309829 7250 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:38:54.312000 7250 webserver.cc:460] Webserver started at http://127.24.76.65:36287/ using document root <none> and password file <none>
I20260808 11:38:54.312651 7250 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:54.312745 7250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:54.313063 7250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:54.315796 7250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/data/instance:
uuid: "7bfe5fd483e04b07b1493c69fa32d9ac"
format_stamp: "Formatted at 2026-08-08 11:38:54 on dist-test-slave-tc2s"
server_key: "9ce29de9155c0a3e096a75abdb382596"
server_key_iv: "cdfee6fa8b9ffeffef980ca6a2a3a6a6"
server_key_version: "encryptionkey@0"
I20260808 11:38:54.316586 7250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance:
uuid: "7bfe5fd483e04b07b1493c69fa32d9ac"
format_stamp: "Formatted at 2026-08-08 11:38:54 on dist-test-slave-tc2s"
server_key: "9ce29de9155c0a3e096a75abdb382596"
server_key_iv: "cdfee6fa8b9ffeffef980ca6a2a3a6a6"
server_key_version: "encryptionkey@0"
I20260808 11:38:54.321403 7250 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260808 11:38:54.324723 7271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:54.325987 7250 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:38:54.326136 7250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/wal
uuid: "7bfe5fd483e04b07b1493c69fa32d9ac"
format_stamp: "Formatted at 2026-08-08 11:38:54 on dist-test-slave-tc2s"
server_key: "9ce29de9155c0a3e096a75abdb382596"
server_key_iv: "cdfee6fa8b9ffeffef980ca6a2a3a6a6"
server_key_version: "encryptionkey@0"
I20260808 11:38:54.326334 7250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:54.342819 7250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:54.343566 7250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:54.343832 7250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:54.344683 7250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:38:54.346043 7250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:38:54.346140 7250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:54.346243 7250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:38:54.346318 7250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:54.376245 7250 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.65:43987
I20260808 11:38:54.376308 7453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.65:43987 every 8 connection(s)
I20260808 11:38:54.377468 7250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
I20260808 11:38:54.385886 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 7250
I20260808 11:38:54.386104 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance
I20260808 11:38:54.386847 24881 external_mini_cluster.cc:1428] Setting key b6c8b7c33f76201423405f81f1120fbc
I20260808 11:38:54.389328 7457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:54.379841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:50731 (local address 127.24.76.126:37811)
0808 11:38:54.380265 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:54.380270 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:54.381402 (+ 1132us) server_negotiation.cc:407] Connection header received
0808 11:38:54.383110 (+ 1708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:54.383114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:54.383180 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:54.383273 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:54.384201 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:54.384923 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:54.386515 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:54.387136 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:54.387271 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:54.388485 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:54.388490 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:54.388493 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:38:54.388510 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:54.388551 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:38:54.388563 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:54.388634 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:54.388909 (+ 275us) server_negotiation.cc:299] Negotiation successful
0808 11:38:54.389005 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":215,"threads_started":1}
I20260808 11:38:54.389755 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.66:0
--local_ip_for_outbound_sockets=127.24.76.66
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:37811
--builtin_ntp_servers=127.24.76.84:38279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 11:38:54.390205 7460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:54.380412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:37811 (local address 127.24.76.65:50731)
0808 11:38:54.381254 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:54.381293 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:54.382890 (+ 1597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:54.383458 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:54.383465 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:54.383547 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:54.384073 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:54.384089 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:54.385033 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:54.385036 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:54.386355 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:54.386373 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:54.387336 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:54.387341 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:54.387538 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:54.388203 (+ 665us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:38:54.388205 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:54.388240 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:38:54.388310 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:54.388663 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:54.388675 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:54.388683 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 11:38:54.388871 (+ 188us) client_negotiation.cc:240] Negotiation successful
0808 11:38:54.389148 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":250,"threads_started":1}
I20260808 11:38:54.391549 7455 heartbeater.cc:347] Connected to a master server at 127.24.76.126:37811
I20260808 11:38:54.391824 7455 heartbeater.cc:464] Registering TS with master...
I20260808 11:38:54.392529 7455 heartbeater.cc:511] Master 127.24.76.126:37811 requested a full tablet report, sending...
I20260808 11:38:54.394466 7153 ts_manager.cc:194] Registered new tserver with Master: 7bfe5fd483e04b07b1493c69fa32d9ac (127.24.76.65:43987)
I20260808 11:38:54.395905 7153 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.76.65:50731
W20260808 11:38:54.714790 7461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:54.715049 7461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:54.715128 7461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:54.722123 7461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:54.722213 7461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:54.722309 7461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.66
I20260808 11:38:54.730949 7461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:38279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:37811
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.66
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:54.732050 7461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:54.733701 7461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:54.746080 7469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:54.746115 7474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:54.746141 7470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:54.746940 7461 server_base.cc:1034] running on GCE node
I20260808 11:38:54.747486 7461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:54.748279 7461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:54.749477 7461 hybrid_clock.cc:648] HybridClock initialized: now 1786189134749423 us; error 32 us; skew 500 ppm
I20260808 11:38:54.749763 7461 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:38:54.751837 7461 webserver.cc:460] Webserver started at http://127.24.76.66:38513/ using document root <none> and password file <none>
I20260808 11:38:54.752468 7461 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:54.752561 7461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:54.752864 7461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:54.755618 7461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/data/instance:
uuid: "fd0b4f8b324f467cb61d48b69ac3961d"
format_stamp: "Formatted at 2026-08-08 11:38:54 on dist-test-slave-tc2s"
server_key: "01cfb25a54f04ef0e8c22fd759d6f862"
server_key_iv: "2b96f6786f6e66563c6820f8cfc1ecdc"
server_key_version: "encryptionkey@0"
I20260808 11:38:54.756403 7461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance:
uuid: "fd0b4f8b324f467cb61d48b69ac3961d"
format_stamp: "Formatted at 2026-08-08 11:38:54 on dist-test-slave-tc2s"
server_key: "01cfb25a54f04ef0e8c22fd759d6f862"
server_key_iv: "2b96f6786f6e66563c6820f8cfc1ecdc"
server_key_version: "encryptionkey@0"
I20260808 11:38:54.761111 7461 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260808 11:38:54.764350 7484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:54.765549 7461 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 11:38:54.765697 7461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/wal
uuid: "fd0b4f8b324f467cb61d48b69ac3961d"
format_stamp: "Formatted at 2026-08-08 11:38:54 on dist-test-slave-tc2s"
server_key: "01cfb25a54f04ef0e8c22fd759d6f862"
server_key_iv: "2b96f6786f6e66563c6820f8cfc1ecdc"
server_key_version: "encryptionkey@0"
I20260808 11:38:54.765883 7461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:54.787082 7461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:54.787813 7461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:54.788075 7461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:54.788924 7461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:38:54.790300 7461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:38:54.790396 7461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:54.790503 7461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:38:54.790573 7461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:54.824358 7461 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.66:45021
I20260808 11:38:54.824437 7660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.66:45021 every 8 connection(s)
I20260808 11:38:54.825641 7461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
I20260808 11:38:54.827327 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 7461
I20260808 11:38:54.827458 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance
I20260808 11:38:54.827873 24881 external_mini_cluster.cc:1428] Setting key 2be598707eda64dac2e805fd73fcd248
I20260808 11:38:54.830665 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.67:0
--local_ip_for_outbound_sockets=127.24.76.67
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:37811
--builtin_ntp_servers=127.24.76.84:38279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 11:38:54.838335 7457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:54.828191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:41743 (local address 127.24.76.126:37811)
0808 11:38:54.828401 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:54.828407 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:54.829949 (+ 1542us) server_negotiation.cc:407] Connection header received
0808 11:38:54.831831 (+ 1882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:54.831837 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:54.831918 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:54.832024 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:54.833007 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:54.833976 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:54.835527 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:54.836011 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:54.836152 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:54.837600 (+ 1448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:54.837605 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:54.837609 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:38:54.837630 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:54.837665 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:38:54.837679 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:54.837767 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:54.837957 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 11:38:54.838053 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 11:38:54.839319 7668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:54.828763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:37811 (local address 127.24.76.66:41743)
0808 11:38:54.829772 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:54.829827 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:54.831571 (+ 1744us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:54.832194 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:54.832202 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:54.832286 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:54.832844 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:54.832861 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:54.834139 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:54.834144 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:54.835380 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:54.835393 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:54.836348 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:54.836357 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:54.836625 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:54.837292 (+ 667us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:38:54.837294 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:54.837335 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:38:54.837418 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:54.837764 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:54.837771 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:54.837777 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 11:38:54.837946 (+ 169us) client_negotiation.cc:240] Negotiation successful
0808 11:38:54.838248 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":315,"threads_started":1}
I20260808 11:38:54.840749 7663 heartbeater.cc:347] Connected to a master server at 127.24.76.126:37811
I20260808 11:38:54.841037 7663 heartbeater.cc:464] Registering TS with master...
I20260808 11:38:54.841784 7663 heartbeater.cc:511] Master 127.24.76.126:37811 requested a full tablet report, sending...
I20260808 11:38:54.843170 7153 ts_manager.cc:194] Registered new tserver with Master: fd0b4f8b324f467cb61d48b69ac3961d (127.24.76.66:45021)
I20260808 11:38:54.843955 7153 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.76.66:41743
W20260808 11:38:55.159163 7669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:55.159425 7669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:55.159544 7669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:55.166421 7669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:55.166508 7669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:55.166646 7669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.67
I20260808 11:38:55.175547 7669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:38279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:37811
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.67
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:55.176779 7669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:55.178547 7669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:55.190961 7674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:55.190886 7680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:55.190924 7678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:55.191637 7669 server_base.cc:1034] running on GCE node
I20260808 11:38:55.192183 7669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:55.193004 7669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:55.194192 7669 hybrid_clock.cc:648] HybridClock initialized: now 1786189135194149 us; error 33 us; skew 500 ppm
I20260808 11:38:55.194478 7669 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:38:55.196563 7669 webserver.cc:460] Webserver started at http://127.24.76.67:42331/ using document root <none> and password file <none>
I20260808 11:38:55.197213 7669 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:55.197312 7669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:55.197609 7669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:55.200317 7669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/data/instance:
uuid: "0cf6878f8e074b2a808eb2322179dfb1"
format_stamp: "Formatted at 2026-08-08 11:38:55 on dist-test-slave-tc2s"
server_key: "5aafa6ad3ede2f4bac9f3d2047b39cb7"
server_key_iv: "0ea14a86009caec1599ed08bf77f4701"
server_key_version: "encryptionkey@0"
I20260808 11:38:55.201095 7669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance:
uuid: "0cf6878f8e074b2a808eb2322179dfb1"
format_stamp: "Formatted at 2026-08-08 11:38:55 on dist-test-slave-tc2s"
server_key: "5aafa6ad3ede2f4bac9f3d2047b39cb7"
server_key_iv: "0ea14a86009caec1599ed08bf77f4701"
server_key_version: "encryptionkey@0"
I20260808 11:38:55.205745 7669 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260808 11:38:55.208884 7691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:55.210062 7669 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 11:38:55.210212 7669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/wal
uuid: "0cf6878f8e074b2a808eb2322179dfb1"
format_stamp: "Formatted at 2026-08-08 11:38:55 on dist-test-slave-tc2s"
server_key: "5aafa6ad3ede2f4bac9f3d2047b39cb7"
server_key_iv: "0ea14a86009caec1599ed08bf77f4701"
server_key_version: "encryptionkey@0"
I20260808 11:38:55.210417 7669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:55.238893 7669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:55.239682 7669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:55.239962 7669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:55.240880 7669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:38:55.242273 7669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:38:55.242368 7669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:55.242476 7669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:38:55.242547 7669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:55.272545 7669 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.67:44563
I20260808 11:38:55.272612 7863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.67:44563 every 8 connection(s)
I20260808 11:38:55.273733 7669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
I20260808 11:38:55.278854 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 7669
I20260808 11:38:55.279003 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance
I20260808 11:38:55.279489 24881 external_mini_cluster.cc:1428] Setting key 70858c8714f4056186b5170a6d99b69d
I20260808 11:38:55.285787 7457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:55.276123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:47419 (local address 127.24.76.126:37811)
0808 11:38:55.276290 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:55.276295 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:55.277613 (+ 1318us) server_negotiation.cc:407] Connection header received
0808 11:38:55.279387 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:55.279394 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:55.279480 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:55.279605 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:55.280744 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:55.281605 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:55.283295 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:55.283711 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:55.283818 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:55.285013 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:55.285018 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:55.285021 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:38:55.285038 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:55.285071 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:38:55.285082 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:55.285155 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:55.285364 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 11:38:55.285459 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 11:38:55.286696 7867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:55.276614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:37811 (local address 127.24.76.67:47419)
0808 11:38:55.277468 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:55.277509 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:55.279134 (+ 1625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:55.279810 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:55.279818 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:55.279908 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:55.280589 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:55.280609 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:55.281752 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:55.281756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:55.283158 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:38:55.283171 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:38:55.283918 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:38:55.283926 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:38:55.284140 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:55.284722 (+ 582us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:38:55.284725 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:55.284760 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:38:55.284833 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:55.285171 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:55.285182 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:55.285189 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 11:38:55.285338 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 11:38:55.285714 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":271,"threads_started":1}
I20260808 11:38:55.287845 7864 heartbeater.cc:347] Connected to a master server at 127.24.76.126:37811
I20260808 11:38:55.288112 7864 heartbeater.cc:464] Registering TS with master...
I20260808 11:38:55.288821 7864 heartbeater.cc:511] Master 127.24.76.126:37811 requested a full tablet report, sending...
I20260808 11:38:55.290141 7153 ts_manager.cc:194] Registered new tserver with Master: 0cf6878f8e074b2a808eb2322179dfb1 (127.24.76.67:44563)
I20260808 11:38:55.290894 7153 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.76.67:47419
I20260808 11:38:55.294451 24881 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 11:38:55.300153 7457 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:55.298720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41436 (local address 127.24.76.126:37811)
0808 11:38:55.298902 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:55.298909 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:55.299318 (+ 409us) server_negotiation.cc:407] Connection header received
0808 11:38:55.299457 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:55.299460 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:55.299539 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:55.299613 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:55.299928 (+ 315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41436: BlockingRecv error: recv got EOF from 127.0.0.1:41436 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 11:38:55.300738 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 7250
I20260808 11:38:55.307934 7884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:55.301642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51842 (local address 127.24.76.65:43987)
0808 11:38:55.302209 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:55.302217 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:55.302241 (+ 24us) server_negotiation.cc:407] Connection header received
0808 11:38:55.302295 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:55.302301 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:55.302484 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:55.302597 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:55.302898 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:55.303846 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:55.305070 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:55.305610 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:55.305706 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:55.306601 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:55.306610 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:55.306622 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:38:55.307208 (+ 586us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:55.307253 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:38:55.307273 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:55.307357 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:55.307444 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 11:38:55.307570 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":292,"threads_started":1}
I20260808 11:38:55.309090 7448 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:55.391215 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 7250
I20260808 11:38:55.416008 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 7461
I20260808 11:38:55.423483 7889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:55.416944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54142 (local address 127.24.76.66:45021)
0808 11:38:55.417439 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:55.417446 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:55.417474 (+ 28us) server_negotiation.cc:407] Connection header received
0808 11:38:55.417536 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:55.417542 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:55.417749 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:55.417886 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:55.418260 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:55.419233 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:55.420492 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:55.421047 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:55.421143 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:55.422152 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:55.422160 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:55.422174 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:38:55.422751 (+ 577us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:55.422801 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:38:55.422821 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:55.422914 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:55.423011 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 11:38:55.423133 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":240,"threads_started":1}
I20260808 11:38:55.424666 7656 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:55.505002 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 7461
I20260808 11:38:55.527063 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 7669
I20260808 11:38:55.535267 7892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:55.528086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51022 (local address 127.24.76.67:44563)
0808 11:38:55.528721 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:55.528728 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:55.528759 (+ 31us) server_negotiation.cc:407] Connection header received
0808 11:38:55.528831 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:55.528837 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:55.529078 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:55.529235 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:55.529718 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:55.530813 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:55.532049 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:38:55.532620 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:38:55.532733 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:38:55.533794 (+ 1061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:55.533802 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:55.533818 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:38:55.534443 (+ 625us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:55.534491 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:38:55.534522 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:55.534625 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:55.534772 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 11:38:55.534915 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":318,"threads_started":1}
I20260808 11:38:55.536494 7859 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:55.618537 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 7669
I20260808 11:38:55.641202 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 7105
I20260808 11:38:55.642434 7203 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:55.744091 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 7105
2026-08-08T11:38:55Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2441 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T11:38:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:38:55Z Disabled control of system clock
I20260808 11:38:55.837575 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:38281
--webserver_interface=127.24.76.126
--webserver_port=0
--builtin_ntp_servers=127.24.76.84:37319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:38281
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 11:38:56.164889 7903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:56.165169 7903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:56.165302 7903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:56.172506 7903 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:38:56.172621 7903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:56.172688 7903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:56.172767 7903 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:38:56.172829 7903 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:38:56.181775 7903 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:37319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:38281
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:38281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:56.182979 7903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:56.184751 7903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:56.195807 7919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:56.195859 7913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:56.195878 7916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:56.196942 7903 server_base.cc:1034] running on GCE node
I20260808 11:38:56.197654 7903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:56.198925 7903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:56.200116 7903 hybrid_clock.cc:648] HybridClock initialized: now 1786189136200071 us; error 43 us; skew 500 ppm
I20260808 11:38:56.200415 7903 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:38:56.202555 7903 webserver.cc:460] Webserver started at http://127.24.76.126:41327/ using document root <none> and password file <none>
I20260808 11:38:56.203193 7903 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:56.203287 7903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:56.203599 7903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:56.206372 7903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "e3f46a2658e549ed91593ab35ea64842"
format_stamp: "Formatted at 2026-08-08 11:38:56 on dist-test-slave-tc2s"
server_key: "952f24efc99b02f3be52aa4188024043"
server_key_iv: "267278d1fe6b6b50f7b2faa73e103deb"
server_key_version: "encryptionkey@0"
I20260808 11:38:56.207130 7903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "e3f46a2658e549ed91593ab35ea64842"
format_stamp: "Formatted at 2026-08-08 11:38:56 on dist-test-slave-tc2s"
server_key: "952f24efc99b02f3be52aa4188024043"
server_key_iv: "267278d1fe6b6b50f7b2faa73e103deb"
server_key_version: "encryptionkey@0"
I20260808 11:38:56.211915 7903 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260808 11:38:56.215193 7929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:56.216367 7903 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.004s
I20260808 11:38:56.216524 7903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "e3f46a2658e549ed91593ab35ea64842"
format_stamp: "Formatted at 2026-08-08 11:38:56 on dist-test-slave-tc2s"
server_key: "952f24efc99b02f3be52aa4188024043"
server_key_iv: "267278d1fe6b6b50f7b2faa73e103deb"
server_key_version: "encryptionkey@0"
I20260808 11:38:56.216710 7903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:56.222108 7903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:56.222833 7903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:56.223093 7903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:56.239621 7903 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:38281
I20260808 11:38:56.239655 8012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:38281 every 8 connection(s)
I20260808 11:38:56.240876 7903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:38:56.243624 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 7903
I20260808 11:38:56.243849 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:38:56.244666 24881 external_mini_cluster.cc:1428] Setting key bf050ec5e3b128d99478806ba2286a69
I20260808 11:38:56.244861 8013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:56.252656 8021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:56.246688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52620 (local address 127.24.76.126:38281)
0808 11:38:56.247633 (+ 945us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:56.247643 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:56.247692 (+ 49us) server_negotiation.cc:407] Connection header received
0808 11:38:56.249385 (+ 1693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:56.249406 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:56.249789 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:56.250141 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:56.251184 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:56.251190 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:56.251201 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:38:56.251215 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:56.251285 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:38:56.251301 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:56.251459 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:56.251569 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 11:38:56.251814 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":340,"threads_started":1}
I20260808 11:38:56.257314 8013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3f46a2658e549ed91593ab35ea64842: Bootstrap starting.
I20260808 11:38:56.260135 8013 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3f46a2658e549ed91593ab35ea64842: Neither blocks nor log segments found. Creating new log.
I20260808 11:38:56.261076 8013 log.cc:824] T 00000000000000000000000000000000 P e3f46a2658e549ed91593ab35ea64842: Log is configured to *not* fsync() on all Append() calls
I20260808 11:38:56.263545 8013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3f46a2658e549ed91593ab35ea64842: No bootstrap required, opened a new log
I20260808 11:38:56.270176 8013 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3f46a2658e549ed91593ab35ea64842 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3f46a2658e549ed91593ab35ea64842" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 38281 } }
I20260808 11:38:56.270503 8013 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3f46a2658e549ed91593ab35ea64842 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:38:56.270604 8013 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3f46a2658e549ed91593ab35ea64842 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3f46a2658e549ed91593ab35ea64842, State: Initialized, Role: FOLLOWER
I20260808 11:38:56.271214 8013 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3f46a2658e549ed91593ab35ea64842 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3f46a2658e549ed91593ab35ea64842" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 38281 } }
I20260808 11:38:56.271471 8013 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3f46a2658e549ed91593ab35ea64842 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:38:56.271612 8013 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3f46a2658e549ed91593ab35ea64842 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:38:56.271763 8013 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3f46a2658e549ed91593ab35ea64842 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:38:56.273764 8013 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3f46a2658e549ed91593ab35ea64842 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3f46a2658e549ed91593ab35ea64842" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 38281 } }
I20260808 11:38:56.274235 8013 leader_election.cc:302] T 00000000000000000000000000000000 P e3f46a2658e549ed91593ab35ea64842 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3f46a2658e549ed91593ab35ea64842; no voters:
I20260808 11:38:56.274794 8013 leader_election.cc:288] T 00000000000000000000000000000000 P e3f46a2658e549ed91593ab35ea64842 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:38:56.274909 8023 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3f46a2658e549ed91593ab35ea64842 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:38:56.275569 8023 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3f46a2658e549ed91593ab35ea64842 [term 1 LEADER]: Becoming Leader. State: Replica: e3f46a2658e549ed91593ab35ea64842, State: Running, Role: LEADER
I20260808 11:38:56.276074 8023 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3f46a2658e549ed91593ab35ea64842 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3f46a2658e549ed91593ab35ea64842" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 38281 } }
I20260808 11:38:56.276605 8013 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3f46a2658e549ed91593ab35ea64842 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:38:56.278452 8024 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3f46a2658e549ed91593ab35ea64842 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3f46a2658e549ed91593ab35ea64842" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3f46a2658e549ed91593ab35ea64842" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 38281 } } }
I20260808 11:38:56.278805 8024 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3f46a2658e549ed91593ab35ea64842 [sys.catalog]: This master's current role is: LEADER
I20260808 11:38:56.282059 8025 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3f46a2658e549ed91593ab35ea64842 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3f46a2658e549ed91593ab35ea64842. Latest consensus state: current_term: 1 leader_uuid: "e3f46a2658e549ed91593ab35ea64842" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3f46a2658e549ed91593ab35ea64842" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 38281 } } }
I20260808 11:38:56.283218 8025 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3f46a2658e549ed91593ab35ea64842 [sys.catalog]: This master's current role is: LEADER
I20260808 11:38:56.283941 8037 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:38:56.288199 8037 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:38:56.296480 8037 catalog_manager.cc:1347] Generated new cluster ID: 9eaebfcb01be4077997776fd93fcec4a
I20260808 11:38:56.296577 8037 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:38:56.315315 8037 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:38:56.315681 8037 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:38:56.330263 8037 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3f46a2658e549ed91593ab35ea64842: Generated new TSK 0
I20260808 11:38:56.331029 8037 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:38:56.346007 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.65:0
--local_ip_for_outbound_sockets=127.24.76.65
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:38281
--builtin_ntp_servers=127.24.76.84:37319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 11:38:56.680258 8056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:56.680536 8056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:56.680662 8056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:56.687889 8056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:56.687979 8056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:56.688130 8056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.65
I20260808 11:38:56.696980 8056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:37319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:38281
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.65
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:56.698180 8056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:56.699905 8056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:56.712244 8062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:56.712082 8065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:56.712294 8069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:56.713227 8056 server_base.cc:1034] running on GCE node
I20260808 11:38:56.713802 8056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:56.714751 8056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:56.715961 8056 hybrid_clock.cc:648] HybridClock initialized: now 1786189136715919 us; error 36 us; skew 500 ppm
I20260808 11:38:56.716228 8056 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:38:56.718407 8056 webserver.cc:460] Webserver started at http://127.24.76.65:40285/ using document root <none> and password file <none>
I20260808 11:38:56.719041 8056 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:56.719123 8056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:56.719388 8056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:56.722285 8056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/data/instance:
uuid: "527cc39be82b4085b5695485556f2b47"
format_stamp: "Formatted at 2026-08-08 11:38:56 on dist-test-slave-tc2s"
server_key: "77e62398474252253d97b0d8cca65875"
server_key_iv: "ccd324e7c379cd7c7a09943af064c5e7"
server_key_version: "encryptionkey@0"
I20260808 11:38:56.723022 8056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance:
uuid: "527cc39be82b4085b5695485556f2b47"
format_stamp: "Formatted at 2026-08-08 11:38:56 on dist-test-slave-tc2s"
server_key: "77e62398474252253d97b0d8cca65875"
server_key_iv: "ccd324e7c379cd7c7a09943af064c5e7"
server_key_version: "encryptionkey@0"
I20260808 11:38:56.727891 8056 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 11:38:56.731125 8081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:56.732353 8056 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:38:56.732506 8056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/wal
uuid: "527cc39be82b4085b5695485556f2b47"
format_stamp: "Formatted at 2026-08-08 11:38:56 on dist-test-slave-tc2s"
server_key: "77e62398474252253d97b0d8cca65875"
server_key_iv: "ccd324e7c379cd7c7a09943af064c5e7"
server_key_version: "encryptionkey@0"
I20260808 11:38:56.732705 8056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:56.738605 8056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:56.739334 8056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:56.739583 8056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:56.740442 8056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:38:56.741765 8056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:38:56.741852 8056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:56.742010 8056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:38:56.742091 8056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:56.771875 8056 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.65:33973
I20260808 11:38:56.771924 8245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.65:33973 every 8 connection(s)
I20260808 11:38:56.773092 8056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
I20260808 11:38:56.778265 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 8056
I20260808 11:38:56.778475 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance
I20260808 11:38:56.779286 24881 external_mini_cluster.cc:1428] Setting key 5dcc09b26d68780f17bd9af2e68c725f
I20260808 11:38:56.781440 8248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:56.775471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:39887 (local address 127.24.76.126:38281)
0808 11:38:56.775894 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:56.775900 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:56.777057 (+ 1157us) server_negotiation.cc:407] Connection header received
0808 11:38:56.779178 (+ 2121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:56.779182 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:56.779238 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:56.779328 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:56.780727 (+ 1399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:56.780733 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:56.780738 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:38:56.780757 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:56.780806 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:38:56.780821 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:56.780934 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:56.781113 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 11:38:56.781176 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":204,"threads_started":1}
I20260808 11:38:56.782445 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.66:0
--local_ip_for_outbound_sockets=127.24.76.66
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:38281
--builtin_ntp_servers=127.24.76.84:37319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 11:38:56.782524 8249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:56.776000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:38281 (local address 127.24.76.65:39887)
0808 11:38:56.776898 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:56.776941 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:56.778876 (+ 1935us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:56.779605 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:56.779615 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:56.779729 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:56.780433 (+ 704us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:38:56.780437 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:56.780481 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:38:56.780564 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:56.780905 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:56.780911 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:56.780916 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 11:38:56.781166 (+ 250us) client_negotiation.cc:240] Negotiation successful
0808 11:38:56.781370 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":243,"threads_started":1}
I20260808 11:38:56.783876 8246 heartbeater.cc:347] Connected to a master server at 127.24.76.126:38281
I20260808 11:38:56.784148 8246 heartbeater.cc:464] Registering TS with master...
I20260808 11:38:56.784701 8246 heartbeater.cc:511] Master 127.24.76.126:38281 requested a full tablet report, sending...
I20260808 11:38:56.786623 7957 ts_manager.cc:194] Registered new tserver with Master: 527cc39be82b4085b5695485556f2b47 (127.24.76.65:33973)
W20260808 11:38:57.108603 8250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:57.108888 8250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:57.109022 8250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:57.116092 8250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:57.116187 8250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:57.116341 8250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.66
I20260808 11:38:57.125212 8250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:37319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:38281
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.66
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:57.126446 8250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:57.128131 8250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:57.140380 8261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:57.140548 8258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:57.140426 8259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:57.141512 8250 server_base.cc:1034] running on GCE node
I20260808 11:38:57.142109 8250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:57.142974 8250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:57.144170 8250 hybrid_clock.cc:648] HybridClock initialized: now 1786189137144128 us; error 31 us; skew 500 ppm
I20260808 11:38:57.144459 8250 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:38:57.146540 8250 webserver.cc:460] Webserver started at http://127.24.76.66:46133/ using document root <none> and password file <none>
I20260808 11:38:57.147152 8250 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:57.147249 8250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:57.147583 8250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:57.150291 8250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/data/instance:
uuid: "2fc70aedd18440a4aa02517ebf340378"
format_stamp: "Formatted at 2026-08-08 11:38:57 on dist-test-slave-tc2s"
server_key: "d912682d072948cc23d705a69118aa21"
server_key_iv: "3608cbf125225826f14f07604ee91416"
server_key_version: "encryptionkey@0"
I20260808 11:38:57.151067 8250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance:
uuid: "2fc70aedd18440a4aa02517ebf340378"
format_stamp: "Formatted at 2026-08-08 11:38:57 on dist-test-slave-tc2s"
server_key: "d912682d072948cc23d705a69118aa21"
server_key_iv: "3608cbf125225826f14f07604ee91416"
server_key_version: "encryptionkey@0"
I20260808 11:38:57.155731 8250 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 11:38:57.158913 8267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:57.160174 8250 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:38:57.160356 8250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/wal
uuid: "2fc70aedd18440a4aa02517ebf340378"
format_stamp: "Formatted at 2026-08-08 11:38:57 on dist-test-slave-tc2s"
server_key: "d912682d072948cc23d705a69118aa21"
server_key_iv: "3608cbf125225826f14f07604ee91416"
server_key_version: "encryptionkey@0"
I20260808 11:38:57.160557 8250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:57.165829 8250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:57.166584 8250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:57.166824 8250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:57.167681 8250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:38:57.168983 8250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:38:57.169077 8250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:57.169178 8250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:38:57.169253 8250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:57.198987 8250 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.66:35439
I20260808 11:38:57.199038 8436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.66:35439 every 8 connection(s)
I20260808 11:38:57.200251 8250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
I20260808 11:38:57.207803 8248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:57.202635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:39679 (local address 127.24.76.126:38281)
0808 11:38:57.202862 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:57.202869 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:57.204152 (+ 1283us) server_negotiation.cc:407] Connection header received
0808 11:38:57.205798 (+ 1646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:57.205802 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:57.205866 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:57.205987 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:57.207171 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:57.207175 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:57.207178 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:38:57.207196 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:57.207229 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:38:57.207256 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:57.207355 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:57.207517 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 11:38:57.207565 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 11:38:57.208173 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 8250
I20260808 11:38:57.208365 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance
I20260808 11:38:57.208858 8444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:57.203124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:38281 (local address 127.24.76.66:39679)
0808 11:38:57.204014 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:57.204049 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:57.205544 (+ 1495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:57.206164 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:57.206172 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:57.206255 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:57.206873 (+ 618us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:38:57.206875 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:57.206911 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:38:57.207001 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:57.207364 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:57.207371 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:57.207376 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 11:38:57.207572 (+ 196us) client_negotiation.cc:240] Negotiation successful
0808 11:38:57.207745 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":295,"threads_started":1}
I20260808 11:38:57.208912 24881 external_mini_cluster.cc:1428] Setting key f33842072d0362e609fd2f8cbb32800b
I20260808 11:38:57.210290 8439 heartbeater.cc:347] Connected to a master server at 127.24.76.126:38281
I20260808 11:38:57.210588 8439 heartbeater.cc:464] Registering TS with master...
I20260808 11:38:57.211133 8439 heartbeater.cc:511] Master 127.24.76.126:38281 requested a full tablet report, sending...
I20260808 11:38:57.211572 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.67:0
--local_ip_for_outbound_sockets=127.24.76.67
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:38281
--builtin_ntp_servers=127.24.76.84:37319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 11:38:57.212497 7957 ts_manager.cc:194] Registered new tserver with Master: 2fc70aedd18440a4aa02517ebf340378 (127.24.76.66:35439)
W20260808 11:38:57.539690 8446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:57.539939 8446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:57.540076 8446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:57.547112 8446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:57.547199 8446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:57.547339 8446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.67
I20260808 11:38:57.556011 8446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:37319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:38281
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.67
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:57.557175 8446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:57.558929 8446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:57.571137 8453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:57.571105 8454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:57.571204 8456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:57.571971 8446 server_base.cc:1034] running on GCE node
I20260808 11:38:57.572500 8446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:57.573354 8446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:57.574544 8446 hybrid_clock.cc:648] HybridClock initialized: now 1786189137574520 us; error 81 us; skew 500 ppm
I20260808 11:38:57.574828 8446 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:38:57.580938 8446 webserver.cc:460] Webserver started at http://127.24.76.67:44811/ using document root <none> and password file <none>
I20260808 11:38:57.581600 8446 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:57.581702 8446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:57.582072 8446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:57.584945 8446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/data/instance:
uuid: "74bd6395cc8f4354bbe9536005814f34"
format_stamp: "Formatted at 2026-08-08 11:38:57 on dist-test-slave-tc2s"
server_key: "a37a32c8f4e447d48882aa2be71d464c"
server_key_iv: "47cfe7574a7b0c59b3b4bda95229150c"
server_key_version: "encryptionkey@0"
I20260808 11:38:57.585711 8446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance:
uuid: "74bd6395cc8f4354bbe9536005814f34"
format_stamp: "Formatted at 2026-08-08 11:38:57 on dist-test-slave-tc2s"
server_key: "a37a32c8f4e447d48882aa2be71d464c"
server_key_iv: "47cfe7574a7b0c59b3b4bda95229150c"
server_key_version: "encryptionkey@0"
I20260808 11:38:57.590545 8446 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260808 11:38:57.593748 8463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:57.594995 8446 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 11:38:57.595119 8446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/wal
uuid: "74bd6395cc8f4354bbe9536005814f34"
format_stamp: "Formatted at 2026-08-08 11:38:57 on dist-test-slave-tc2s"
server_key: "a37a32c8f4e447d48882aa2be71d464c"
server_key_iv: "47cfe7574a7b0c59b3b4bda95229150c"
server_key_version: "encryptionkey@0"
I20260808 11:38:57.595291 8446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:57.600448 8446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:57.601140 8446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:57.601346 8446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:57.602183 8446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:38:57.603444 8446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:38:57.603528 8446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:57.603605 8446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:38:57.603657 8446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:57.632797 8446 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.67:39749
I20260808 11:38:57.632846 8632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.67:39749 every 8 connection(s)
I20260808 11:38:57.634050 8446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
I20260808 11:38:57.638141 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 8446
I20260808 11:38:57.638329 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance
I20260808 11:38:57.638867 24881 external_mini_cluster.cc:1428] Setting key 895018e2dece6dfea2a88001cd376c66
I20260808 11:38:57.642258 8248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:57.636434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:57641 (local address 127.24.76.126:38281)
0808 11:38:57.636601 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:57.636605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:57.638006 (+ 1401us) server_negotiation.cc:407] Connection header received
0808 11:38:57.640092 (+ 2086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:57.640099 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:57.640166 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:57.640256 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:57.641575 (+ 1319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:57.641579 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:57.641582 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:38:57.641598 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:57.641630 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:38:57.641642 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:57.641777 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:57.641990 (+ 213us) server_negotiation.cc:299] Negotiation successful
0808 11:38:57.642041 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 11:38:57.643116 8637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:57.636886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:38281 (local address 127.24.76.67:57641)
0808 11:38:57.637821 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0808 11:38:57.637876 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 11:38:57.639804 (+ 1928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:38:57.640455 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:38:57.640463 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:38:57.640554 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:38:57.641238 (+ 684us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:38:57.641241 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:38:57.641290 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:38:57.641408 (+ 118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:38:57.641749 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:38:57.641759 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:38:57.641768 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 11:38:57.642081 (+ 313us) client_negotiation.cc:240] Negotiation successful
0808 11:38:57.642286 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":271,"threads_started":1}
I20260808 11:38:57.644189 8633 heartbeater.cc:347] Connected to a master server at 127.24.76.126:38281
I20260808 11:38:57.644421 8633 heartbeater.cc:464] Registering TS with master...
I20260808 11:38:57.644914 8633 heartbeater.cc:511] Master 127.24.76.126:38281 requested a full tablet report, sending...
I20260808 11:38:57.646127 7957 ts_manager.cc:194] Registered new tserver with Master: 74bd6395cc8f4354bbe9536005814f34 (127.24.76.67:39749)
I20260808 11:38:57.653949 24881 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 11:38:57.659827 8248 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:38:57.658431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52634 (local address 127.24.76.126:38281)
0808 11:38:57.658580 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:57.658584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:57.659094 (+ 510us) server_negotiation.cc:407] Connection header received
0808 11:38:57.659213 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:57.659216 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:57.659266 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:57.659332 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:57.659591 (+ 259us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52634: BlockingRecv error: recv got EOF from 127.0.0.1:52634 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 11:38:57.660452 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 8056
I20260808 11:38:57.663735 8646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:57.661412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60214 (local address 127.24.76.65:33973)
0808 11:38:57.661979 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:57.661991 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:57.662017 (+ 26us) server_negotiation.cc:407] Connection header received
0808 11:38:57.662072 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:57.662077 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:57.662253 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:57.662366 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:57.663153 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:57.663159 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:57.663169 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:38:57.663186 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:57.663222 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:38:57.663235 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:57.663335 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:57.663382 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 11:38:57.663455 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":265,"threads_started":1}
I20260808 11:38:57.664856 8239 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:57.747512 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 8056
I20260808 11:38:57.766585 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 8250
I20260808 11:38:57.769816 8648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:57.767424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49252 (local address 127.24.76.66:35439)
0808 11:38:57.767934 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:57.767941 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:57.767970 (+ 29us) server_negotiation.cc:407] Connection header received
0808 11:38:57.768036 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:57.768042 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:57.768252 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:57.768387 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:57.769208 (+ 821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:57.769214 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:57.769225 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:38:57.769236 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:57.769273 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:38:57.769290 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:57.769400 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:57.769447 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 11:38:57.769536 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":223,"threads_started":1}
I20260808 11:38:57.771020 8426 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:57.850481 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 8250
I20260808 11:38:57.872566 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 8446
I20260808 11:38:57.876557 8652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:57.873560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58532 (local address 127.24.76.67:39749)
0808 11:38:57.874202 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:57.874209 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:57.874241 (+ 32us) server_negotiation.cc:407] Connection header received
0808 11:38:57.874304 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:57.874311 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:57.874526 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:57.874663 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:57.875816 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:38:57.875824 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:38:57.875835 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:38:57.875852 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:38:57.875889 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:38:57.875902 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:38:57.876066 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 11:38:57.876156 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 11:38:57.876259 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":318,"threads_started":1}
I20260808 11:38:57.877785 8627 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:57.956605 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 8446
I20260808 11:38:57.978811 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 7903
I20260808 11:38:57.979948 8004 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:38:58.081475 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 7903
2026-08-08T11:38:58Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2338 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:38:58 dist-test-slave-tc2s krb5kdc[8660](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:38:58 dist-test-slave-tc2s krb5kdc[8660](info): set up 2 sockets
Aug 08 11:38:58 dist-test-slave-tc2s krb5kdc[8660](info): commencing operation
krb5kdc: starting...
W20260808 11:38:59.300611 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.001s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:38:59 dist-test-slave-tc2s krb5kdc[8660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:38:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:38:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
I20260808 11:38:59.559080 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:38367
--webserver_interface=127.24.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:33149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:38367
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:38:59.884610 8680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:38:59.884884 8680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:38:59.885022 8680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:38:59.892014 8680 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:38:59.892108 8680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:38:59.892185 8680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:38:59.892261 8680 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:38:59.892323 8680 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:38:59.901752 8680 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:33149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:38367
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:38367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:38:59.902978 8680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:38:59.904219 8680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:38:59.915114 8689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:59.915223 8690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:38:59.915331 8692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:38:59.916350 8680 server_base.cc:1034] running on GCE node
I20260808 11:38:59.917073 8680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:38:59.918394 8680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:38:59.919601 8680 hybrid_clock.cc:648] HybridClock initialized: now 1786189139919549 us; error 30 us; skew 500 ppm
I20260808 11:38:59.919931 8680 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:38:59 dist-test-slave-tc2s krb5kdc[8660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189139, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:38:59.922912 8680 init.cc:377] Logged in from keytab as kudu/127.24.76.126@KRBTEST.COM (short username kudu)
I20260808 11:38:59.924213 8680 webserver.cc:460] Webserver started at http://127.24.76.126:40681/ using document root <none> and password file <none>
I20260808 11:38:59.924836 8680 fs_manager.cc:359] Metadata directory not provided
I20260808 11:38:59.924947 8680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:38:59.925236 8680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:38:59.928066 8680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "62e1723e853847b885b059dda22a5f18"
format_stamp: "Formatted at 2026-08-08 11:38:59 on dist-test-slave-tc2s"
server_key: "e4f58b24b4dd6e72028034d413af6de9"
server_key_iv: "9375dff7fdeae2e4471f7bf18e3e9c2a"
server_key_version: "encryptionkey@0"
I20260808 11:38:59.928778 8680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "62e1723e853847b885b059dda22a5f18"
format_stamp: "Formatted at 2026-08-08 11:38:59 on dist-test-slave-tc2s"
server_key: "e4f58b24b4dd6e72028034d413af6de9"
server_key_iv: "9375dff7fdeae2e4471f7bf18e3e9c2a"
server_key_version: "encryptionkey@0"
I20260808 11:38:59.933573 8680 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260808 11:38:59.936710 8705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:38:59.937870 8680 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 11:38:59.938071 8680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "62e1723e853847b885b059dda22a5f18"
format_stamp: "Formatted at 2026-08-08 11:38:59 on dist-test-slave-tc2s"
server_key: "e4f58b24b4dd6e72028034d413af6de9"
server_key_iv: "9375dff7fdeae2e4471f7bf18e3e9c2a"
server_key_version: "encryptionkey@0"
I20260808 11:38:59.938237 8680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:38:59.965010 8680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:38:59.969120 8680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:38:59.969391 8680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:38:59.986202 8680 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:38367
I20260808 11:38:59.986241 8781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:38367 every 8 connection(s)
I20260808 11:38:59.987444 8680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:38:59.990978 8782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:38:59.992097 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 8680
I20260808 11:38:59.992302 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:38:59.992929 24881 external_mini_cluster.cc:1428] Setting key cedfa10e9ef7445828aa1efe398547c3
I20260808 11:39:00.004527 8782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62e1723e853847b885b059dda22a5f18: Bootstrap starting.
Aug 08 11:39:00 dist-test-slave-tc2s krb5kdc[8660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:39:00.007396 8782 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62e1723e853847b885b059dda22a5f18: Neither blocks nor log segments found. Creating new log.
I20260808 11:39:00.008654 8782 log.cc:824] T 00000000000000000000000000000000 P 62e1723e853847b885b059dda22a5f18: Log is configured to *not* fsync() on all Append() calls
I20260808 11:39:00.012019 8782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62e1723e853847b885b059dda22a5f18: No bootstrap required, opened a new log
I20260808 11:39:00.015949 8787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:38:59.994881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55136 (local address 127.24.76.126:38367)
0808 11:38:59.995721 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0808 11:38:59.995736 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 11:38:59.995794 (+ 58us) server_negotiation.cc:407] Connection header received
0808 11:38:59.997392 (+ 1598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:38:59.997418 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:38:59.997928 (+ 510us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:38:59.998338 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:38:59.999649 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:00.001652 (+ 2003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:00.002957 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:00.003944 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:00.004135 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:00.007830 (+ 3695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:39:00.007841 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:39:00.007863 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:39:00.007912 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:39:00.011686 (+ 3774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:39:00.012354 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:39:00.012363 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:39:00.012376 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:39:00.012603 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:39:00.012926 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:39:00.012931 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:39:00.012934 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:39:00.013594 (+ 660us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:39:00.013946 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0808 11:39:00.014248 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 11:39:00.014685 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":341,"threads_started":1}
I20260808 11:39:00.021061 8782 raft_consensus.cc:348] T 00000000000000000000000000000000 P 62e1723e853847b885b059dda22a5f18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e1723e853847b885b059dda22a5f18" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 38367 } }
I20260808 11:39:00.021412 8782 raft_consensus.cc:374] T 00000000000000000000000000000000 P 62e1723e853847b885b059dda22a5f18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:39:00.021530 8782 raft_consensus.cc:729] T 00000000000000000000000000000000 P 62e1723e853847b885b059dda22a5f18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62e1723e853847b885b059dda22a5f18, State: Initialized, Role: FOLLOWER
I20260808 11:39:00.022116 8782 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62e1723e853847b885b059dda22a5f18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e1723e853847b885b059dda22a5f18" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 38367 } }
I20260808 11:39:00.022328 8782 raft_consensus.cc:388] T 00000000000000000000000000000000 P 62e1723e853847b885b059dda22a5f18 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:39:00.022454 8782 raft_consensus.cc:482] T 00000000000000000000000000000000 P 62e1723e853847b885b059dda22a5f18 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:39:00.022586 8782 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 62e1723e853847b885b059dda22a5f18 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:39:00.024577 8782 raft_consensus.cc:504] T 00000000000000000000000000000000 P 62e1723e853847b885b059dda22a5f18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e1723e853847b885b059dda22a5f18" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 38367 } }
I20260808 11:39:00.025019 8782 leader_election.cc:302] T 00000000000000000000000000000000 P 62e1723e853847b885b059dda22a5f18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62e1723e853847b885b059dda22a5f18; no voters:
I20260808 11:39:00.025529 8782 leader_election.cc:288] T 00000000000000000000000000000000 P 62e1723e853847b885b059dda22a5f18 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:39:00.025653 8792 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 62e1723e853847b885b059dda22a5f18 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:39:00.026307 8792 raft_consensus.cc:686] T 00000000000000000000000000000000 P 62e1723e853847b885b059dda22a5f18 [term 1 LEADER]: Becoming Leader. State: Replica: 62e1723e853847b885b059dda22a5f18, State: Running, Role: LEADER
I20260808 11:39:00.026873 8792 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62e1723e853847b885b059dda22a5f18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e1723e853847b885b059dda22a5f18" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 38367 } }
I20260808 11:39:00.027395 8782 sys_catalog.cc:564] T 00000000000000000000000000000000 P 62e1723e853847b885b059dda22a5f18 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:39:00.029311 8794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62e1723e853847b885b059dda22a5f18 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "62e1723e853847b885b059dda22a5f18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e1723e853847b885b059dda22a5f18" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 38367 } } }
I20260808 11:39:00.029589 8794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62e1723e853847b885b059dda22a5f18 [sys.catalog]: This master's current role is: LEADER
I20260808 11:39:00.030645 8795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62e1723e853847b885b059dda22a5f18 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62e1723e853847b885b059dda22a5f18. Latest consensus state: current_term: 1 leader_uuid: "62e1723e853847b885b059dda22a5f18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62e1723e853847b885b059dda22a5f18" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 38367 } } }
I20260808 11:39:00.031026 8795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62e1723e853847b885b059dda22a5f18 [sys.catalog]: This master's current role is: LEADER
I20260808 11:39:00.031056 8798 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:39:00.036228 8798 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:39:00.045132 8798 catalog_manager.cc:1347] Generated new cluster ID: 541e7b46f5ef47119cbdfca1432e1794
I20260808 11:39:00.045228 8798 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:39:00.069523 8798 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:39:00.070791 8798 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:39:00.079348 8798 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 62e1723e853847b885b059dda22a5f18: Generated new TSK 0
I20260808 11:39:00.080068 8798 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab.
Entry for principal kudu/127.24.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab.
I20260808 11:39:00.165616 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.65:0
--local_ip_for_outbound_sockets=127.24.76.65
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:38367
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:33149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:39:00.499557 8823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:39:00.499833 8823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:39:00.499974 8823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:39:00.507112 8823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:39:00.507223 8823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:39:00.507375 8823 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.65
I20260808 11:39:00.517256 8823 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:33149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:38367
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.65
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:39:00.518517 8823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:39:00.519747 8823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:39:00.532315 8830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:39:00.532254 8834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:39:00.532225 8838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:39:00.533030 8823 server_base.cc:1034] running on GCE node
I20260808 11:39:00.533617 8823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:39:00.534579 8823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:39:00.535801 8823 hybrid_clock.cc:648] HybridClock initialized: now 1786189140535732 us; error 56 us; skew 500 ppm
I20260808 11:39:00.536108 8823 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:39:00 dist-test-slave-tc2s krb5kdc[8660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189140, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:39:00.539258 8823 init.cc:377] Logged in from keytab as kudu/127.24.76.65@KRBTEST.COM (short username kudu)
I20260808 11:39:00.540580 8823 webserver.cc:460] Webserver started at http://127.24.76.65:39299/ using document root <none> and password file <none>
I20260808 11:39:00.541234 8823 fs_manager.cc:359] Metadata directory not provided
I20260808 11:39:00.541360 8823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:39:00.541666 8823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:39:00.544553 8823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/data/instance:
uuid: "54d7207f5bd3480ab41cde754b3fdf8d"
format_stamp: "Formatted at 2026-08-08 11:39:00 on dist-test-slave-tc2s"
server_key: "e7d2b5616d383bb975e7c2713fd592c1"
server_key_iv: "3b18b42159453e5dce0d21792c9e8fad"
server_key_version: "encryptionkey@0"
I20260808 11:39:00.545271 8823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance:
uuid: "54d7207f5bd3480ab41cde754b3fdf8d"
format_stamp: "Formatted at 2026-08-08 11:39:00 on dist-test-slave-tc2s"
server_key: "e7d2b5616d383bb975e7c2713fd592c1"
server_key_iv: "3b18b42159453e5dce0d21792c9e8fad"
server_key_version: "encryptionkey@0"
I20260808 11:39:00.550151 8823 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260808 11:39:00.553360 8850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:39:00.554661 8823 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:39:00.554796 8823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/wal
uuid: "54d7207f5bd3480ab41cde754b3fdf8d"
format_stamp: "Formatted at 2026-08-08 11:39:00 on dist-test-slave-tc2s"
server_key: "e7d2b5616d383bb975e7c2713fd592c1"
server_key_iv: "3b18b42159453e5dce0d21792c9e8fad"
server_key_version: "encryptionkey@0"
I20260808 11:39:00.554979 8823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:39:00.569878 8823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:39:00.573777 8823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:39:00.574067 8823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:39:00.574898 8823 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:39:00.576113 8823 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:39:00.576182 8823 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:39:00.576287 8823 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:39:00.576330 8823 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:39:00.606230 8823 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.65:45577
I20260808 11:39:00.606277 9025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.65:45577 every 8 connection(s)
I20260808 11:39:00.607419 8823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
I20260808 11:39:00.612854 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 8823
I20260808 11:39:00.613058 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance
I20260808 11:39:00.613677 24881 external_mini_cluster.cc:1428] Setting key cdf89f4b471211935fcde85b15ffb8eb
Aug 08 11:39:00 dist-test-slave-tc2s krb5kdc[8660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189140, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:39:00.626259 9030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:00.609749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:48853 (local address 127.24.76.126:38367)
0808 11:39:00.610220 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0808 11:39:00.610225 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:39:00.611137 (+ 912us) server_negotiation.cc:407] Connection header received
0808 11:39:00.612924 (+ 1787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:39:00.612930 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:39:00.613017 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:39:00.613114 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:39:00.614724 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:00.615569 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:00.617108 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:00.617500 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:00.617609 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:00.621078 (+ 3469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:39:00.621083 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:39:00.621086 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:39:00.621116 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:39:00.623409 (+ 2293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:39:00.624069 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:39:00.624073 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:39:00.624076 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:39:00.624194 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:39:00.624596 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:39:00.624600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:39:00.624602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:39:00.624861 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:39:00.625001 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 11:39:00.625664 (+ 663us) server_negotiation.cc:299] Negotiation successful
0808 11:39:00.625883 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":238,"threads_started":1}
I20260808 11:39:00.627050 9031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:00.610288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:38367 (local address 127.24.76.65:48853)
0808 11:39:00.610980 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0808 11:39:00.611022 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 11:39:00.612664 (+ 1642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:39:00.613361 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:39:00.613368 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:39:00.613870 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:39:00.614556 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:39:00.614578 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:39:00.615725 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:39:00.615729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:39:00.616965 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:39:00.616978 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:39:00.617714 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:39:00.617721 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:39:00.618015 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:00.618648 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:39:00.618670 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:39:00.620869 (+ 2199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:39:00.623534 (+ 2665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:39:00.623543 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:39:00.623558 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:39:00.623952 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:39:00.624293 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:39:00.624298 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:39:00.624302 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:39:00.624487 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:39:00.624982 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:39:00.624989 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:39:00.625285 (+ 296us) client_negotiation.cc:769] Sending connection context
0808 11:39:00.625515 (+ 230us) client_negotiation.cc:240] Negotiation successful
0808 11:39:00.626010 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":267,"threads_started":1}
I20260808 11:39:00.628475 9026 heartbeater.cc:347] Connected to a master server at 127.24.76.126:38367
I20260808 11:39:00.628748 9026 heartbeater.cc:464] Registering TS with master...
I20260808 11:39:00.629467 9026 heartbeater.cc:511] Master 127.24.76.126:38367 requested a full tablet report, sending...
I20260808 11:39:00.631454 8731 ts_manager.cc:194] Registered new tserver with Master: 54d7207f5bd3480ab41cde754b3fdf8d (127.24.76.65:45577)
I20260808 11:39:00.632920 8731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.65@KRBTEST.COM'} at 127.24.76.65:48853
WARNING: no policy specified for kudu/127.24.76.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab.
Entry for principal kudu/127.24.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab.
I20260808 11:39:00.671025 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.66:0
--local_ip_for_outbound_sockets=127.24.76.66
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:38367
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:33149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:39:01.004329 9036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:39:01.004685 9036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:39:01.004835 9036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:39:01.013142 9036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:39:01.013238 9036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:39:01.013360 9036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.66
I20260808 11:39:01.022671 9036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:33149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:38367
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.66
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:39:01.023823 9036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:39:01.025033 9036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:39:01.037528 9045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:39:01.037503 9043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:39:01.038028 9042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:39:01.038312 9036 server_base.cc:1034] running on GCE node
I20260808 11:39:01.038847 9036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:39:01.039644 9036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:39:01.040843 9036 hybrid_clock.cc:648] HybridClock initialized: now 1786189141040792 us; error 48 us; skew 500 ppm
I20260808 11:39:01.041129 9036 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:39:01 dist-test-slave-tc2s krb5kdc[8660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189141, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:39:01.044288 9036 init.cc:377] Logged in from keytab as kudu/127.24.76.66@KRBTEST.COM (short username kudu)
I20260808 11:39:01.045588 9036 webserver.cc:460] Webserver started at http://127.24.76.66:34441/ using document root <none> and password file <none>
I20260808 11:39:01.046267 9036 fs_manager.cc:359] Metadata directory not provided
I20260808 11:39:01.046380 9036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:39:01.046651 9036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:39:01.049362 9036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/data/instance:
uuid: "17801faf2cc3422abe0ea69df1978b80"
format_stamp: "Formatted at 2026-08-08 11:39:01 on dist-test-slave-tc2s"
server_key: "6bc52350dcd23358b3ba9041863d83e8"
server_key_iv: "2f6b5caf9ec49f7959e19933045df27b"
server_key_version: "encryptionkey@0"
I20260808 11:39:01.050117 9036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance:
uuid: "17801faf2cc3422abe0ea69df1978b80"
format_stamp: "Formatted at 2026-08-08 11:39:01 on dist-test-slave-tc2s"
server_key: "6bc52350dcd23358b3ba9041863d83e8"
server_key_iv: "2f6b5caf9ec49f7959e19933045df27b"
server_key_version: "encryptionkey@0"
I20260808 11:39:01.054864 9036 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 11:39:01.058073 9055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:39:01.059295 9036 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 11:39:01.059441 9036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/wal
uuid: "17801faf2cc3422abe0ea69df1978b80"
format_stamp: "Formatted at 2026-08-08 11:39:01 on dist-test-slave-tc2s"
server_key: "6bc52350dcd23358b3ba9041863d83e8"
server_key_iv: "2f6b5caf9ec49f7959e19933045df27b"
server_key_version: "encryptionkey@0"
I20260808 11:39:01.059617 9036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:39:01.073594 9036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:39:01.077389 9036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:39:01.077628 9036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:39:01.078469 9036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:39:01.079730 9036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:39:01.079813 9036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:39:01.079909 9036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:39:01.079959 9036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:39:01.108636 9036 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.66:46387
I20260808 11:39:01.108676 9218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.66:46387 every 8 connection(s)
I20260808 11:39:01.109835 9036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
I20260808 11:39:01.114466 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 9036
I20260808 11:39:01.114634 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance
I20260808 11:39:01.115051 24881 external_mini_cluster.cc:1428] Setting key 41ef097af6f819729990ba6bac17a9c2
Aug 08 11:39:01 dist-test-slave-tc2s krb5kdc[8660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189141, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:39:01.129261 9030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:01.112247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:34629 (local address 127.24.76.126:38367)
0808 11:39:01.112401 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 11:39:01.112405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:39:01.113547 (+ 1142us) server_negotiation.cc:407] Connection header received
0808 11:39:01.115373 (+ 1826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:39:01.115380 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:39:01.115475 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:39:01.115589 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:39:01.117107 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:01.118107 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:01.119567 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:01.120118 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:01.120237 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:01.123401 (+ 3164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:39:01.123407 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:39:01.123411 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:39:01.123454 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:39:01.126327 (+ 2873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:39:01.126953 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:39:01.126959 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:39:01.126962 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:39:01.127117 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:39:01.127511 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:39:01.127515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:39:01.127518 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:39:01.127837 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:39:01.127989 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 11:39:01.128694 (+ 705us) server_negotiation.cc:299] Negotiation successful
0808 11:39:01.128922 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 11:39:01.130220 9221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:01.112723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:38367 (local address 127.24.76.66:34629)
0808 11:39:01.113400 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0808 11:39:01.113435 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 11:39:01.115116 (+ 1681us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:39:01.115804 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:39:01.115816 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:39:01.116327 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:39:01.116932 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:39:01.116958 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:39:01.118229 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:39:01.118233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:39:01.119427 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:39:01.119439 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:39:01.120306 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:39:01.120312 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:39:01.120505 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:01.121105 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:39:01.121127 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:39:01.123189 (+ 2062us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:39:01.126451 (+ 3262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:39:01.126457 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:39:01.126468 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:39:01.126845 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:39:01.127216 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:39:01.127219 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:39:01.127221 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:39:01.127419 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:39:01.127969 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:39:01.127978 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:39:01.128312 (+ 334us) client_negotiation.cc:769] Sending connection context
0808 11:39:01.128571 (+ 259us) client_negotiation.cc:240] Negotiation successful
0808 11:39:01.129043 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":266,"threads_started":1}
I20260808 11:39:01.131551 9219 heartbeater.cc:347] Connected to a master server at 127.24.76.126:38367
I20260808 11:39:01.131882 9219 heartbeater.cc:464] Registering TS with master...
I20260808 11:39:01.132669 9219 heartbeater.cc:511] Master 127.24.76.126:38367 requested a full tablet report, sending...
I20260808 11:39:01.134255 8731 ts_manager.cc:194] Registered new tserver with Master: 17801faf2cc3422abe0ea69df1978b80 (127.24.76.66:46387)
I20260808 11:39:01.135306 8731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.66@KRBTEST.COM'} at 127.24.76.66:34629
WARNING: no policy specified for kudu/127.24.76.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab.
Entry for principal kudu/127.24.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab.
I20260808 11:39:01.176752 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.67:0
--local_ip_for_outbound_sockets=127.24.76.67
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:38367
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:33149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:39:01.504549 9225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:39:01.504812 9225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:39:01.504936 9225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:39:01.511801 9225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:39:01.511911 9225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:39:01.512061 9225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.67
I20260808 11:39:01.521200 9225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:33149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:38367
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.67
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:39:01.522413 9225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:39:01.523631 9225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:39:01.536064 9234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:39:01.536041 9238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:39:01.536001 9235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:39:01.536844 9225 server_base.cc:1034] running on GCE node
I20260808 11:39:01.537405 9225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:39:01.538302 9225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:39:01.539500 9225 hybrid_clock.cc:648] HybridClock initialized: now 1786189141539455 us; error 34 us; skew 500 ppm
I20260808 11:39:01.539781 9225 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:39:01 dist-test-slave-tc2s krb5kdc[8660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189141, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:39:01.543059 9225 init.cc:377] Logged in from keytab as kudu/127.24.76.67@KRBTEST.COM (short username kudu)
I20260808 11:39:01.544427 9225 webserver.cc:460] Webserver started at http://127.24.76.67:33435/ using document root <none> and password file <none>
I20260808 11:39:01.545086 9225 fs_manager.cc:359] Metadata directory not provided
I20260808 11:39:01.545194 9225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:39:01.545485 9225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:39:01.548296 9225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/data/instance:
uuid: "76f307f37f0343449b5237dea72a3980"
format_stamp: "Formatted at 2026-08-08 11:39:01 on dist-test-slave-tc2s"
server_key: "1bfe6dc955d56238c7fcbe9124bf5f06"
server_key_iv: "e6b37d60139edd47a4507a764c20d351"
server_key_version: "encryptionkey@0"
I20260808 11:39:01.549026 9225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance:
uuid: "76f307f37f0343449b5237dea72a3980"
format_stamp: "Formatted at 2026-08-08 11:39:01 on dist-test-slave-tc2s"
server_key: "1bfe6dc955d56238c7fcbe9124bf5f06"
server_key_iv: "e6b37d60139edd47a4507a764c20d351"
server_key_version: "encryptionkey@0"
I20260808 11:39:01.553794 9225 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260808 11:39:01.557060 9254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:39:01.558434 9225 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:39:01.558568 9225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/wal
uuid: "76f307f37f0343449b5237dea72a3980"
format_stamp: "Formatted at 2026-08-08 11:39:01 on dist-test-slave-tc2s"
server_key: "1bfe6dc955d56238c7fcbe9124bf5f06"
server_key_iv: "e6b37d60139edd47a4507a764c20d351"
server_key_version: "encryptionkey@0"
I20260808 11:39:01.558740 9225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:39:01.576102 9225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:39:01.579648 9225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:39:01.579881 9225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:39:01.580755 9225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:39:01.582033 9225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:39:01.582106 9225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:39:01.582190 9225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:39:01.582231 9225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:39:01.610857 9225 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.67:42073
I20260808 11:39:01.610908 9436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.67:42073 every 8 connection(s)
I20260808 11:39:01.612105 9225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
I20260808 11:39:01.622259 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 9225
I20260808 11:39:01.622457 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance
I20260808 11:39:01.622879 24881 external_mini_cluster.cc:1428] Setting key 31d447e37fff4812edd694bb0e95752c
Aug 08 11:39:01 dist-test-slave-tc2s krb5kdc[8660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189141, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:39:01.630861 9030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:01.614578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:44091 (local address 127.24.76.126:38367)
0808 11:39:01.614760 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 11:39:01.614765 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:39:01.615793 (+ 1028us) server_negotiation.cc:407] Connection header received
0808 11:39:01.617572 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:39:01.617578 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:39:01.617655 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:39:01.617742 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:39:01.619194 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:01.619969 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:01.621509 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:01.622089 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:01.622221 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:01.625994 (+ 3773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:39:01.626001 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:39:01.626005 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:39:01.626040 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:39:01.628449 (+ 2409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:39:01.629050 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:39:01.629054 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:39:01.629056 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:39:01.629171 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:39:01.629502 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:39:01.629505 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:39:01.629507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:39:01.629763 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:39:01.629869 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 11:39:01.630470 (+ 601us) server_negotiation.cc:299] Negotiation successful
0808 11:39:01.630630 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 11:39:01.631851 9441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:01.614967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:38367 (local address 127.24.76.67:44091)
0808 11:39:01.615637 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0808 11:39:01.615682 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 11:39:01.617306 (+ 1624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:39:01.618016 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:39:01.618025 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:39:01.618495 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:39:01.619029 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:39:01.619047 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:39:01.620127 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:39:01.620131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:39:01.621368 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:39:01.621381 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:39:01.622306 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:39:01.622311 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:39:01.622582 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:01.623208 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:39:01.623245 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:39:01.625763 (+ 2518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:39:01.628559 (+ 2796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:39:01.628565 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:39:01.628576 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:39:01.628967 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:39:01.629255 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:39:01.629258 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:39:01.629260 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:39:01.629426 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:39:01.629849 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:39:01.629854 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:39:01.630169 (+ 315us) client_negotiation.cc:769] Sending connection context
0808 11:39:01.630382 (+ 213us) client_negotiation.cc:240] Negotiation successful
0808 11:39:01.630738 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":251,"threads_started":1}
I20260808 11:39:01.633203 9439 heartbeater.cc:347] Connected to a master server at 127.24.76.126:38367
I20260808 11:39:01.633489 9439 heartbeater.cc:464] Registering TS with master...
I20260808 11:39:01.634217 9439 heartbeater.cc:511] Master 127.24.76.126:38367 requested a full tablet report, sending...
I20260808 11:39:01.635502 9026 heartbeater.cc:503] Master 127.24.76.126:38367 was elected leader, sending a full tablet report...
I20260808 11:39:01.635550 8731 ts_manager.cc:194] Registered new tserver with Master: 76f307f37f0343449b5237dea72a3980 (127.24.76.67:42073)
I20260808 11:39:01.636575 8731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.67@KRBTEST.COM'} at 127.24.76.67:44091
I20260808 11:39:01.638268 24881 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:39:01.653371 9030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:01.642579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55146 (local address 127.24.76.126:38367)
0808 11:39:01.642710 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 11:39:01.642714 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:39:01.643150 (+ 436us) server_negotiation.cc:407] Connection header received
0808 11:39:01.643280 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:39:01.643283 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:39:01.643338 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:39:01.643404 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:39:01.644639 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:01.645323 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:01.646517 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:01.646916 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:01.647013 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:01.648961 (+ 1948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:39:01.648966 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:39:01.648969 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:39:01.649003 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:39:01.651380 (+ 2377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:39:01.651919 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:39:01.651923 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:39:01.651925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:39:01.652051 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:39:01.652319 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:39:01.652322 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:39:01.652324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:39:01.652595 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:39:01.652775 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 11:39:01.653019 (+ 244us) server_negotiation.cc:299] Negotiation successful
0808 11:39:01.653166 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 11:39:01.657096 8729 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55146:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:39:01.664024 8729 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:39:01.685604 9458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:01.675872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51076 (local address 127.24.76.66:46387)
0808 11:39:01.676409 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0808 11:39:01.676420 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:39:01.677533 (+ 1113us) server_negotiation.cc:407] Connection header received
0808 11:39:01.677955 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:39:01.677963 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:39:01.678221 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:39:01.678383 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:39:01.679219 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:01.680623 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:01.683111 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:01.684495 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:01.684625 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:01.684814 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0808 11:39:01.685151 (+ 337us) server_negotiation.cc:299] Negotiation successful
0808 11:39:01.685317 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":294,"threads_started":1}
I20260808 11:39:01.685977 9457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:01.675390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:46387 (local address 127.0.0.1:51076)
0808 11:39:01.677407 (+ 2017us) negotiation.cc:107] Waiting for socket to connect
0808 11:39:01.677429 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 11:39:01.677674 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:39:01.678678 (+ 1004us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:39:01.678685 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:39:01.678714 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:39:01.679042 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:39:01.679054 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:39:01.680773 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:39:01.680778 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:39:01.682941 (+ 2163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:39:01.682956 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:39:01.684704 (+ 1748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:39:01.684711 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:39:01.684896 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:01.684947 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 11:39:01.685094 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 11:39:01.685274 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1877,"mutex_wait_us":2,"thread_start_us":1741,"threads_started":1}
I20260808 11:39:01.686389 9461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:01.676156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38934 (local address 127.24.76.65:45577)
0808 11:39:01.679132 (+ 2976us) server_negotiation.cc:206] Beginning negotiation
0808 11:39:01.679140 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:39:01.679164 (+ 24us) server_negotiation.cc:407] Connection header received
0808 11:39:01.679248 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:39:01.679255 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:39:01.679461 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:39:01.679654 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:39:01.680231 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:01.681671 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:01.683673 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:01.685008 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:01.685125 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:01.685329 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0808 11:39:01.685955 (+ 626us) server_negotiation.cc:299] Negotiation successful
0808 11:39:01.686106 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2812,"thread_start_us":2648,"threads_started":1}
I20260808 11:39:01.686434 9460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:01.675781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:45577 (local address 127.0.0.1:38934)
0808 11:39:01.678446 (+ 2665us) negotiation.cc:107] Waiting for socket to connect
0808 11:39:01.678470 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 11:39:01.678583 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:39:01.679656 (+ 1073us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:39:01.679660 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:39:01.679685 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:39:01.680032 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:39:01.680043 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:39:01.681796 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:39:01.681800 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:39:01.683491 (+ 1691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:39:01.683506 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:39:01.685513 (+ 2007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:39:01.685519 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:39:01.685752 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:01.685782 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 11:39:01.685867 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 11:39:01.686020 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2548,"mutex_wait_us":29,"thread_start_us":2419,"threads_started":1}
I20260808 11:39:01.687345 9455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:01.675065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:42073 (local address 127.0.0.1:49808)
0808 11:39:01.675827 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0808 11:39:01.678416 (+ 2589us) client_negotiation.cc:174] Beginning negotiation
0808 11:39:01.678554 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:39:01.679180 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:39:01.679184 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:39:01.679207 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:39:01.679554 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:39:01.679564 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:39:01.681524 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:39:01.681530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:39:01.683504 (+ 1974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:39:01.683536 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:39:01.685573 (+ 2037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:39:01.685585 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:39:01.685882 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:01.685938 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 11:39:01.686031 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 11:39:01.686158 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"mutex_wait_us":66,"thread_start_us":381,"threads_started":1}
I20260808 11:39:01.687672 9456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:01.675278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49808 (local address 127.24.76.67:42073)
0808 11:39:01.676957 (+ 1679us) server_negotiation.cc:206] Beginning negotiation
0808 11:39:01.676966 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:39:01.678675 (+ 1709us) server_negotiation.cc:407] Connection header received
0808 11:39:01.678783 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:39:01.678791 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:39:01.679014 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:39:01.679173 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:39:01.679762 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:01.681366 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:01.683711 (+ 2345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:01.685007 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:01.685120 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:01.685329 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 11:39:01.687196 (+ 1867us) server_negotiation.cc:299] Negotiation successful
0808 11:39:01.687349 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1506,"thread_start_us":1344,"threads_started":1}
I20260808 11:39:01.689759 8929 tablet_service.cc:1467] Processing CreateTablet for tablet 3b3cebe82bba41b7b36fed7d95d9d47c (DEFAULT_TABLE table=test-table [id=244abad25b4645e99b1747baf4af9940]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:39:01.689874 9133 tablet_service.cc:1467] Processing CreateTablet for tablet 3b3cebe82bba41b7b36fed7d95d9d47c (DEFAULT_TABLE table=test-table [id=244abad25b4645e99b1747baf4af9940]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:39:01.691004 8929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b3cebe82bba41b7b36fed7d95d9d47c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:39:01.691263 9133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b3cebe82bba41b7b36fed7d95d9d47c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:39:01.690798 9334 tablet_service.cc:1467] Processing CreateTablet for tablet 3b3cebe82bba41b7b36fed7d95d9d47c (DEFAULT_TABLE table=test-table [id=244abad25b4645e99b1747baf4af9940]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:39:01.692056 9334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b3cebe82bba41b7b36fed7d95d9d47c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:39:01.701777 9462 tablet_bootstrap.cc:492] T 3b3cebe82bba41b7b36fed7d95d9d47c P 54d7207f5bd3480ab41cde754b3fdf8d: Bootstrap starting.
I20260808 11:39:01.704836 9462 tablet_bootstrap.cc:654] T 3b3cebe82bba41b7b36fed7d95d9d47c P 54d7207f5bd3480ab41cde754b3fdf8d: Neither blocks nor log segments found. Creating new log.
I20260808 11:39:01.704918 9466 tablet_bootstrap.cc:492] T 3b3cebe82bba41b7b36fed7d95d9d47c P 17801faf2cc3422abe0ea69df1978b80: Bootstrap starting.
I20260808 11:39:01.706274 9462 log.cc:824] T 3b3cebe82bba41b7b36fed7d95d9d47c P 54d7207f5bd3480ab41cde754b3fdf8d: Log is configured to *not* fsync() on all Append() calls
I20260808 11:39:01.707758 9467 tablet_bootstrap.cc:492] T 3b3cebe82bba41b7b36fed7d95d9d47c P 76f307f37f0343449b5237dea72a3980: Bootstrap starting.
I20260808 11:39:01.708251 9466 tablet_bootstrap.cc:654] T 3b3cebe82bba41b7b36fed7d95d9d47c P 17801faf2cc3422abe0ea69df1978b80: Neither blocks nor log segments found. Creating new log.
I20260808 11:39:01.709154 9462 tablet_bootstrap.cc:492] T 3b3cebe82bba41b7b36fed7d95d9d47c P 54d7207f5bd3480ab41cde754b3fdf8d: No bootstrap required, opened a new log
I20260808 11:39:01.709388 9462 ts_tablet_manager.cc:1397] T 3b3cebe82bba41b7b36fed7d95d9d47c P 54d7207f5bd3480ab41cde754b3fdf8d: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260808 11:39:01.709465 9466 log.cc:824] T 3b3cebe82bba41b7b36fed7d95d9d47c P 17801faf2cc3422abe0ea69df1978b80: Log is configured to *not* fsync() on all Append() calls
I20260808 11:39:01.711035 9467 tablet_bootstrap.cc:654] T 3b3cebe82bba41b7b36fed7d95d9d47c P 76f307f37f0343449b5237dea72a3980: Neither blocks nor log segments found. Creating new log.
I20260808 11:39:01.712188 9467 log.cc:824] T 3b3cebe82bba41b7b36fed7d95d9d47c P 76f307f37f0343449b5237dea72a3980: Log is configured to *not* fsync() on all Append() calls
I20260808 11:39:01.712397 9466 tablet_bootstrap.cc:492] T 3b3cebe82bba41b7b36fed7d95d9d47c P 17801faf2cc3422abe0ea69df1978b80: No bootstrap required, opened a new log
I20260808 11:39:01.712648 9466 ts_tablet_manager.cc:1397] T 3b3cebe82bba41b7b36fed7d95d9d47c P 17801faf2cc3422abe0ea69df1978b80: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260808 11:39:01.715006 9467 tablet_bootstrap.cc:492] T 3b3cebe82bba41b7b36fed7d95d9d47c P 76f307f37f0343449b5237dea72a3980: No bootstrap required, opened a new log
I20260808 11:39:01.715291 9467 ts_tablet_manager.cc:1397] T 3b3cebe82bba41b7b36fed7d95d9d47c P 76f307f37f0343449b5237dea72a3980: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20260808 11:39:01.718451 9462 raft_consensus.cc:348] T 3b3cebe82bba41b7b36fed7d95d9d47c P 54d7207f5bd3480ab41cde754b3fdf8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76f307f37f0343449b5237dea72a3980" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 42073 } } peers { permanent_uuid: "17801faf2cc3422abe0ea69df1978b80" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 46387 } } peers { permanent_uuid: "54d7207f5bd3480ab41cde754b3fdf8d" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 45577 } }
I20260808 11:39:01.718863 9462 raft_consensus.cc:374] T 3b3cebe82bba41b7b36fed7d95d9d47c P 54d7207f5bd3480ab41cde754b3fdf8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:39:01.718972 9462 raft_consensus.cc:729] T 3b3cebe82bba41b7b36fed7d95d9d47c P 54d7207f5bd3480ab41cde754b3fdf8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54d7207f5bd3480ab41cde754b3fdf8d, State: Initialized, Role: FOLLOWER
I20260808 11:39:01.719578 9462 consensus_queue.cc:260] T 3b3cebe82bba41b7b36fed7d95d9d47c P 54d7207f5bd3480ab41cde754b3fdf8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76f307f37f0343449b5237dea72a3980" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 42073 } } peers { permanent_uuid: "17801faf2cc3422abe0ea69df1978b80" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 46387 } } peers { permanent_uuid: "54d7207f5bd3480ab41cde754b3fdf8d" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 45577 } }
I20260808 11:39:01.719779 9466 raft_consensus.cc:348] T 3b3cebe82bba41b7b36fed7d95d9d47c P 17801faf2cc3422abe0ea69df1978b80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76f307f37f0343449b5237dea72a3980" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 42073 } } peers { permanent_uuid: "17801faf2cc3422abe0ea69df1978b80" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 46387 } } peers { permanent_uuid: "54d7207f5bd3480ab41cde754b3fdf8d" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 45577 } }
I20260808 11:39:01.720193 9466 raft_consensus.cc:374] T 3b3cebe82bba41b7b36fed7d95d9d47c P 17801faf2cc3422abe0ea69df1978b80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:39:01.720311 9466 raft_consensus.cc:729] T 3b3cebe82bba41b7b36fed7d95d9d47c P 17801faf2cc3422abe0ea69df1978b80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17801faf2cc3422abe0ea69df1978b80, State: Initialized, Role: FOLLOWER
I20260808 11:39:01.720956 9466 consensus_queue.cc:260] T 3b3cebe82bba41b7b36fed7d95d9d47c P 17801faf2cc3422abe0ea69df1978b80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76f307f37f0343449b5237dea72a3980" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 42073 } } peers { permanent_uuid: "17801faf2cc3422abe0ea69df1978b80" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 46387 } } peers { permanent_uuid: "54d7207f5bd3480ab41cde754b3fdf8d" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 45577 } }
I20260808 11:39:01.721309 9462 ts_tablet_manager.cc:1428] T 3b3cebe82bba41b7b36fed7d95d9d47c P 54d7207f5bd3480ab41cde754b3fdf8d: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260808 11:39:01.722820 9219 heartbeater.cc:503] Master 127.24.76.126:38367 was elected leader, sending a full tablet report...
I20260808 11:39:01.724730 9466 ts_tablet_manager.cc:1428] T 3b3cebe82bba41b7b36fed7d95d9d47c P 17801faf2cc3422abe0ea69df1978b80: Time spent starting tablet: real 0.012s user 0.012s sys 0.001s
I20260808 11:39:01.724924 9467 raft_consensus.cc:348] T 3b3cebe82bba41b7b36fed7d95d9d47c P 76f307f37f0343449b5237dea72a3980 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76f307f37f0343449b5237dea72a3980" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 42073 } } peers { permanent_uuid: "17801faf2cc3422abe0ea69df1978b80" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 46387 } } peers { permanent_uuid: "54d7207f5bd3480ab41cde754b3fdf8d" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 45577 } }
I20260808 11:39:01.725330 9467 raft_consensus.cc:374] T 3b3cebe82bba41b7b36fed7d95d9d47c P 76f307f37f0343449b5237dea72a3980 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:39:01.725433 9467 raft_consensus.cc:729] T 3b3cebe82bba41b7b36fed7d95d9d47c P 76f307f37f0343449b5237dea72a3980 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76f307f37f0343449b5237dea72a3980, State: Initialized, Role: FOLLOWER
I20260808 11:39:01.726070 9467 consensus_queue.cc:260] T 3b3cebe82bba41b7b36fed7d95d9d47c P 76f307f37f0343449b5237dea72a3980 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76f307f37f0343449b5237dea72a3980" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 42073 } } peers { permanent_uuid: "17801faf2cc3422abe0ea69df1978b80" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 46387 } } peers { permanent_uuid: "54d7207f5bd3480ab41cde754b3fdf8d" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 45577 } }
I20260808 11:39:01.727556 9439 heartbeater.cc:503] Master 127.24.76.126:38367 was elected leader, sending a full tablet report...
I20260808 11:39:01.727656 9467 ts_tablet_manager.cc:1428] T 3b3cebe82bba41b7b36fed7d95d9d47c P 76f307f37f0343449b5237dea72a3980: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260808 11:39:01.815220 9473 raft_consensus.cc:482] T 3b3cebe82bba41b7b36fed7d95d9d47c P 17801faf2cc3422abe0ea69df1978b80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:39:01.815541 9473 raft_consensus.cc:504] T 3b3cebe82bba41b7b36fed7d95d9d47c P 17801faf2cc3422abe0ea69df1978b80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76f307f37f0343449b5237dea72a3980" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 42073 } } peers { permanent_uuid: "17801faf2cc3422abe0ea69df1978b80" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 46387 } } peers { permanent_uuid: "54d7207f5bd3480ab41cde754b3fdf8d" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 45577 } }
I20260808 11:39:01.817185 9473 leader_election.cc:288] T 3b3cebe82bba41b7b36fed7d95d9d47c P 17801faf2cc3422abe0ea69df1978b80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76f307f37f0343449b5237dea72a3980 (127.24.76.67:42073), 54d7207f5bd3480ab41cde754b3fdf8d (127.24.76.65:45577)
I20260808 11:39:01.823916 9456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:01.817743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:59323 (local address 127.24.76.67:42073)
0808 11:39:01.817973 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 11:39:01.817980 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:39:01.818751 (+ 771us) server_negotiation.cc:407] Connection header received
0808 11:39:01.818914 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:39:01.818919 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:39:01.818976 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:39:01.819061 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:39:01.819722 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:01.820470 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:01.822205 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:01.822955 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:01.823124 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:01.823197 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 11:39:01.823546 (+ 349us) server_negotiation.cc:299] Negotiation successful
0808 11:39:01.823677 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 11:39:01.823932 9461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:01.817713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:40101 (local address 127.24.76.65:45577)
0808 11:39:01.817962 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0808 11:39:01.817968 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:39:01.818666 (+ 698us) server_negotiation.cc:407] Connection header received
0808 11:39:01.818848 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:39:01.818852 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:39:01.818911 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:39:01.818986 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:39:01.819552 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:01.820271 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:01.822099 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:01.822899 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:01.823013 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:01.823067 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 11:39:01.823561 (+ 494us) server_negotiation.cc:299] Negotiation successful
0808 11:39:01.823689 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 11:39:01.824102 9477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:01.817674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:42073 (local address 127.24.76.66:59323)
0808 11:39:01.818646 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0808 11:39:01.818660 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 11:39:01.818782 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:39:01.819098 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:39:01.819102 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:39:01.819212 (+ 110us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:39:01.819581 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:39:01.819590 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:39:01.820611 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:39:01.820615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:39:01.822042 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:39:01.822057 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:39:01.823196 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:39:01.823203 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:39:01.823384 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:01.823414 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 11:39:01.823504 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 11:39:01.823663 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":877,"mutex_wait_us":28,"spinlock_wait_cycles":34944,"thread_start_us":547,"threads_started":1}
I20260808 11:39:01.824200 9476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:01.817645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:45577 (local address 127.24.76.66:40101)
0808 11:39:01.818556 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0808 11:39:01.818573 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:39:01.818715 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:39:01.819014 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:39:01.819018 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:39:01.819048 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:39:01.819402 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:39:01.819412 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:39:01.820407 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:39:01.820412 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:39:01.821913 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:39:01.821928 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:39:01.823231 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:39:01.823235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:39:01.823405 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:01.823427 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 11:39:01.823503 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 11:39:01.823646 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":775,"mutex_wait_us":48,"spinlock_wait_cycles":25984,"thread_start_us":536,"threads_started":1}
I20260808 11:39:01.825142 9363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b3cebe82bba41b7b36fed7d95d9d47c" candidate_uuid: "17801faf2cc3422abe0ea69df1978b80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76f307f37f0343449b5237dea72a3980" is_pre_election: true
I20260808 11:39:01.825142 8959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b3cebe82bba41b7b36fed7d95d9d47c" candidate_uuid: "17801faf2cc3422abe0ea69df1978b80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54d7207f5bd3480ab41cde754b3fdf8d" is_pre_election: true
I20260808 11:39:01.825634 9363 raft_consensus.cc:2454] T 3b3cebe82bba41b7b36fed7d95d9d47c P 76f307f37f0343449b5237dea72a3980 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17801faf2cc3422abe0ea69df1978b80 in term 0.
I20260808 11:39:01.825632 8959 raft_consensus.cc:2454] T 3b3cebe82bba41b7b36fed7d95d9d47c P 54d7207f5bd3480ab41cde754b3fdf8d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17801faf2cc3422abe0ea69df1978b80 in term 0.
I20260808 11:39:01.826290 9056 leader_election.cc:302] T 3b3cebe82bba41b7b36fed7d95d9d47c P 17801faf2cc3422abe0ea69df1978b80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17801faf2cc3422abe0ea69df1978b80, 76f307f37f0343449b5237dea72a3980; no voters:
I20260808 11:39:01.826701 9473 raft_consensus.cc:2781] T 3b3cebe82bba41b7b36fed7d95d9d47c P 17801faf2cc3422abe0ea69df1978b80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:39:01.826857 9473 raft_consensus.cc:482] T 3b3cebe82bba41b7b36fed7d95d9d47c P 17801faf2cc3422abe0ea69df1978b80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:39:01.826960 9473 raft_consensus.cc:3037] T 3b3cebe82bba41b7b36fed7d95d9d47c P 17801faf2cc3422abe0ea69df1978b80 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:39:01.829334 9473 raft_consensus.cc:504] T 3b3cebe82bba41b7b36fed7d95d9d47c P 17801faf2cc3422abe0ea69df1978b80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76f307f37f0343449b5237dea72a3980" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 42073 } } peers { permanent_uuid: "17801faf2cc3422abe0ea69df1978b80" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 46387 } } peers { permanent_uuid: "54d7207f5bd3480ab41cde754b3fdf8d" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 45577 } }
I20260808 11:39:01.829963 9473 leader_election.cc:288] T 3b3cebe82bba41b7b36fed7d95d9d47c P 17801faf2cc3422abe0ea69df1978b80 [CANDIDATE]: Term 1 election: Requested vote from peers 76f307f37f0343449b5237dea72a3980 (127.24.76.67:42073), 54d7207f5bd3480ab41cde754b3fdf8d (127.24.76.65:45577)
I20260808 11:39:01.830403 9363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b3cebe82bba41b7b36fed7d95d9d47c" candidate_uuid: "17801faf2cc3422abe0ea69df1978b80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76f307f37f0343449b5237dea72a3980"
I20260808 11:39:01.830423 8959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b3cebe82bba41b7b36fed7d95d9d47c" candidate_uuid: "17801faf2cc3422abe0ea69df1978b80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54d7207f5bd3480ab41cde754b3fdf8d"
I20260808 11:39:01.830579 9363 raft_consensus.cc:3037] T 3b3cebe82bba41b7b36fed7d95d9d47c P 76f307f37f0343449b5237dea72a3980 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:39:01.830616 8959 raft_consensus.cc:3037] T 3b3cebe82bba41b7b36fed7d95d9d47c P 54d7207f5bd3480ab41cde754b3fdf8d [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:39:01.832783 8959 raft_consensus.cc:2454] T 3b3cebe82bba41b7b36fed7d95d9d47c P 54d7207f5bd3480ab41cde754b3fdf8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17801faf2cc3422abe0ea69df1978b80 in term 1.
I20260808 11:39:01.832818 9363 raft_consensus.cc:2454] T 3b3cebe82bba41b7b36fed7d95d9d47c P 76f307f37f0343449b5237dea72a3980 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17801faf2cc3422abe0ea69df1978b80 in term 1.
I20260808 11:39:01.833164 9058 leader_election.cc:302] T 3b3cebe82bba41b7b36fed7d95d9d47c P 17801faf2cc3422abe0ea69df1978b80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17801faf2cc3422abe0ea69df1978b80, 54d7207f5bd3480ab41cde754b3fdf8d; no voters:
I20260808 11:39:01.833488 9473 raft_consensus.cc:2781] T 3b3cebe82bba41b7b36fed7d95d9d47c P 17801faf2cc3422abe0ea69df1978b80 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:39:01.833909 9473 raft_consensus.cc:686] T 3b3cebe82bba41b7b36fed7d95d9d47c P 17801faf2cc3422abe0ea69df1978b80 [term 1 LEADER]: Becoming Leader. State: Replica: 17801faf2cc3422abe0ea69df1978b80, State: Running, Role: LEADER
I20260808 11:39:01.834369 9473 consensus_queue.cc:237] T 3b3cebe82bba41b7b36fed7d95d9d47c P 17801faf2cc3422abe0ea69df1978b80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76f307f37f0343449b5237dea72a3980" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 42073 } } peers { permanent_uuid: "17801faf2cc3422abe0ea69df1978b80" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 46387 } } peers { permanent_uuid: "54d7207f5bd3480ab41cde754b3fdf8d" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 45577 } }
I20260808 11:39:01.839254 8731 catalog_manager.cc:5495] T 3b3cebe82bba41b7b36fed7d95d9d47c P 17801faf2cc3422abe0ea69df1978b80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 17801faf2cc3422abe0ea69df1978b80 (127.24.76.66). New cstate: current_term: 1 leader_uuid: "17801faf2cc3422abe0ea69df1978b80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76f307f37f0343449b5237dea72a3980" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 42073 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17801faf2cc3422abe0ea69df1978b80" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 46387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54d7207f5bd3480ab41cde754b3fdf8d" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 45577 } health_report { overall_health: UNKNOWN } } }
W20260808 11:39:01.859611 9028 tablet.cc:2369] T 3b3cebe82bba41b7b36fed7d95d9d47c P 54d7207f5bd3480ab41cde754b3fdf8d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:39:01.861625 9220 tablet.cc:2369] T 3b3cebe82bba41b7b36fed7d95d9d47c P 17801faf2cc3422abe0ea69df1978b80: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:39:01.863498 9440 tablet.cc:2369] T 3b3cebe82bba41b7b36fed7d95d9d47c P 76f307f37f0343449b5237dea72a3980: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:39:01.875934 9458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:01.870631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51086 (local address 127.24.76.66:46387)
0808 11:39:01.870828 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 11:39:01.870834 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:39:01.870969 (+ 135us) server_negotiation.cc:407] Connection header received
0808 11:39:01.871197 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:39:01.871201 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:39:01.871277 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:39:01.871403 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:39:01.871805 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:01.872527 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:01.873834 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:01.874285 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:01.874396 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:01.874877 (+ 481us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:39:01.875363 (+ 486us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:39:01.875520 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 11:39:01.875595 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 11:39:01.875676 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 11:39:01.883646 9363 raft_consensus.cc:1261] T 3b3cebe82bba41b7b36fed7d95d9d47c P 76f307f37f0343449b5237dea72a3980 [term 1 FOLLOWER]: Refusing update from remote peer 17801faf2cc3422abe0ea69df1978b80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:39:01.883647 8959 raft_consensus.cc:1261] T 3b3cebe82bba41b7b36fed7d95d9d47c P 54d7207f5bd3480ab41cde754b3fdf8d [term 1 FOLLOWER]: Refusing update from remote peer 17801faf2cc3422abe0ea69df1978b80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:39:01.884487 9478 consensus_queue.cc:1035] T 3b3cebe82bba41b7b36fed7d95d9d47c P 17801faf2cc3422abe0ea69df1978b80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54d7207f5bd3480ab41cde754b3fdf8d" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 45577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:39:01.884851 9473 consensus_queue.cc:1035] T 3b3cebe82bba41b7b36fed7d95d9d47c P 17801faf2cc3422abe0ea69df1978b80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76f307f37f0343449b5237dea72a3980" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 42073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:39:01.897254 9485 mvcc.cc:205] Tried to move back new op lower bound from 7316230725145513984 to 7316230724957560832. Current Snapshot: MvccSnapshot[applied={T|T < 7316230725145513984}]
I20260808 11:39:01.898136 9486 mvcc.cc:205] Tried to move back new op lower bound from 7316230725145513984 to 7316230724957560832. Current Snapshot: MvccSnapshot[applied={T|T < 7316230725145513984}]
I20260808 11:39:01.907136 8731 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:39:01.907377 8731 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:39:01.910684 8731 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 62e1723e853847b885b059dda22a5f18: Sending DeleteTablet for 3 replicas of tablet 3b3cebe82bba41b7b36fed7d95d9d47c
I20260808 11:39:01.911743 9334 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b3cebe82bba41b7b36fed7d95d9d47c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:39:01 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:49808
I20260808 11:39:01.911952 9133 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b3cebe82bba41b7b36fed7d95d9d47c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:39:01 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:51076
I20260808 11:39:01.912379 8929 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b3cebe82bba41b7b36fed7d95d9d47c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:39:01 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:38934
I20260808 11:39:01.913427 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 8823
I20260808 11:39:01.916404 9499 tablet_replica.cc:330] T 3b3cebe82bba41b7b36fed7d95d9d47c P 54d7207f5bd3480ab41cde754b3fdf8d: stopping tablet replica
I20260808 11:39:01.916785 9499 raft_consensus.cc:2229] T 3b3cebe82bba41b7b36fed7d95d9d47c P 54d7207f5bd3480ab41cde754b3fdf8d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:39:01.917246 9499 raft_consensus.cc:2258] T 3b3cebe82bba41b7b36fed7d95d9d47c P 54d7207f5bd3480ab41cde754b3fdf8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:39:01.919260 9499 ts_tablet_manager.cc:1905] T 3b3cebe82bba41b7b36fed7d95d9d47c P 54d7207f5bd3480ab41cde754b3fdf8d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:39:01.926327 9499 ts_tablet_manager.cc:1918] T 3b3cebe82bba41b7b36fed7d95d9d47c P 54d7207f5bd3480ab41cde754b3fdf8d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:39:01.926914 9497 tablet_replica.cc:330] T 3b3cebe82bba41b7b36fed7d95d9d47c P 17801faf2cc3422abe0ea69df1978b80: stopping tablet replica
I20260808 11:39:01.927187 9499 log.cc:1196] T 3b3cebe82bba41b7b36fed7d95d9d47c P 54d7207f5bd3480ab41cde754b3fdf8d: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-0/wal/wals/3b3cebe82bba41b7b36fed7d95d9d47c
I20260808 11:39:01.927323 9497 raft_consensus.cc:2229] T 3b3cebe82bba41b7b36fed7d95d9d47c P 17801faf2cc3422abe0ea69df1978b80 [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:39:01.927908 9499 ts_tablet_manager.cc:1939] T 3b3cebe82bba41b7b36fed7d95d9d47c P 54d7207f5bd3480ab41cde754b3fdf8d: Deleting consensus metadata
I20260808 11:39:01.928009 9497 raft_consensus.cc:2258] T 3b3cebe82bba41b7b36fed7d95d9d47c P 17801faf2cc3422abe0ea69df1978b80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:39:01.930001 9497 ts_tablet_manager.cc:1905] T 3b3cebe82bba41b7b36fed7d95d9d47c P 17801faf2cc3422abe0ea69df1978b80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:39:01.930322 8708 catalog_manager.cc:4841] TS 54d7207f5bd3480ab41cde754b3fdf8d (127.24.76.65:45577): tablet 3b3cebe82bba41b7b36fed7d95d9d47c (table test-table [id=244abad25b4645e99b1747baf4af9940]) successfully deleted
I20260808 11:39:01.931687 9496 tablet_replica.cc:330] T 3b3cebe82bba41b7b36fed7d95d9d47c P 76f307f37f0343449b5237dea72a3980: stopping tablet replica
I20260808 11:39:01.932137 9496 raft_consensus.cc:2229] T 3b3cebe82bba41b7b36fed7d95d9d47c P 76f307f37f0343449b5237dea72a3980 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:39:01.932667 9496 raft_consensus.cc:2258] T 3b3cebe82bba41b7b36fed7d95d9d47c P 76f307f37f0343449b5237dea72a3980 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:39:01.934779 9496 ts_tablet_manager.cc:1905] T 3b3cebe82bba41b7b36fed7d95d9d47c P 76f307f37f0343449b5237dea72a3980: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:39:01.937132 9497 ts_tablet_manager.cc:1918] T 3b3cebe82bba41b7b36fed7d95d9d47c P 17801faf2cc3422abe0ea69df1978b80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:39:01.937292 9497 log.cc:1196] T 3b3cebe82bba41b7b36fed7d95d9d47c P 17801faf2cc3422abe0ea69df1978b80: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-1/wal/wals/3b3cebe82bba41b7b36fed7d95d9d47c
I20260808 11:39:01.937744 9497 ts_tablet_manager.cc:1939] T 3b3cebe82bba41b7b36fed7d95d9d47c P 17801faf2cc3422abe0ea69df1978b80: Deleting consensus metadata
I20260808 11:39:01.939430 8712 catalog_manager.cc:4841] TS 17801faf2cc3422abe0ea69df1978b80 (127.24.76.66:46387): tablet 3b3cebe82bba41b7b36fed7d95d9d47c (table test-table [id=244abad25b4645e99b1747baf4af9940]) successfully deleted
Aug 08 11:39:01 dist-test-slave-tc2s krb5kdc[8660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.65@KRBTEST.COM
I20260808 11:39:01.942169 9496 ts_tablet_manager.cc:1918] T 3b3cebe82bba41b7b36fed7d95d9d47c P 76f307f37f0343449b5237dea72a3980: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:39:01.942324 9496 log.cc:1196] T 3b3cebe82bba41b7b36fed7d95d9d47c P 76f307f37f0343449b5237dea72a3980: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786189042529715-24881-0/minicluster-data/ts-2/wal/wals/3b3cebe82bba41b7b36fed7d95d9d47c
I20260808 11:39:01.943186 9496 ts_tablet_manager.cc:1939] T 3b3cebe82bba41b7b36fed7d95d9d47c P 76f307f37f0343449b5237dea72a3980: Deleting consensus metadata
I20260808 11:39:01.944856 8707 catalog_manager.cc:4841] TS 76f307f37f0343449b5237dea72a3980 (127.24.76.67:42073): tablet 3b3cebe82bba41b7b36fed7d95d9d47c (table test-table [id=244abad25b4645e99b1747baf4af9940]) successfully deleted
I20260808 11:39:01.947741 9461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:01.914519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38946 (local address 127.24.76.65:45577)
0808 11:39:01.914679 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 11:39:01.914685 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:39:01.931085 (+ 16400us) server_negotiation.cc:407] Connection header received
0808 11:39:01.931266 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:39:01.931272 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:39:01.931346 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:39:01.931438 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:39:01.933227 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:01.934290 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:01.936110 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:01.936720 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:01.936815 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:01.941975 (+ 5160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:39:01.941989 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:39:01.942002 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:39:01.942042 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:39:01.944963 (+ 2921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:39:01.945625 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:39:01.945635 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:39:01.945638 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:39:01.945808 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:39:01.946182 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:39:01.946187 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:39:01.946191 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:39:01.946658 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:39:01.946963 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0808 11:39:01.947146 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 11:39:01.947369 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 11:39:01.948395 9020 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:39:02.036566 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 8823
I20260808 11:39:02.070554 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 9036
Aug 08 11:39:02 dist-test-slave-tc2s krb5kdc[8660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.66@KRBTEST.COM
I20260808 11:39:02.087445 9458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:02.071441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51096 (local address 127.24.76.66:46387)
0808 11:39:02.071618 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 11:39:02.071623 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:39:02.071641 (+ 18us) server_negotiation.cc:407] Connection header received
0808 11:39:02.071781 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:39:02.071785 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:39:02.071866 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:39:02.071951 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:39:02.073528 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:02.074549 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:02.076311 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:02.076734 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:02.076828 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:02.081791 (+ 4963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:39:02.081810 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:39:02.081829 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:39:02.081866 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:39:02.084783 (+ 2917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:39:02.085535 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:39:02.085542 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:39:02.085545 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:39:02.085682 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:39:02.086087 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:39:02.086091 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:39:02.086093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:39:02.086446 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:39:02.086708 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0808 11:39:02.086955 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 11:39:02.087114 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 11:39:02.088222 9212 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:39:02.200682 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 9036
I20260808 11:39:02.224924 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 9225
Aug 08 11:39:02 dist-test-slave-tc2s krb5kdc[8660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.67@KRBTEST.COM
I20260808 11:39:02.239856 9456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:02.225750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49824 (local address 127.24.76.67:42073)
0808 11:39:02.225958 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 11:39:02.225964 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:39:02.226001 (+ 37us) server_negotiation.cc:407] Connection header received
0808 11:39:02.226138 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:39:02.226143 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:39:02.226214 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:39:02.226296 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:39:02.227705 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:02.228542 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:02.229808 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:02.230378 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:02.230492 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:02.234676 (+ 4184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:39:02.234691 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:39:02.234708 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:39:02.234741 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:39:02.237235 (+ 2494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:39:02.237856 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:39:02.237864 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:39:02.237866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:39:02.238048 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:39:02.238383 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:39:02.238389 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:39:02.238392 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:39:02.238751 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:39:02.239054 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0808 11:39:02.239354 (+ 300us) server_negotiation.cc:299] Negotiation successful
0808 11:39:02.239559 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 11:39:02.240492 9425 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:39:02.328497 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 9225
I20260808 11:39:02.354997 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 8680
I20260808 11:39:02.356083 8773 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:39:02.466305 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 8680
2026-08-08T11:39:02Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4384 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:39:02 dist-test-slave-tc2s krb5kdc[9512](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:39:02 dist-test-slave-tc2s krb5kdc[9512](info): set up 2 sockets
Aug 08 11:39:02 dist-test-slave-tc2s krb5kdc[9512](info): commencing operation
krb5kdc: starting...
W20260808 11:39:04.204571 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.691s user 0.003s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:39:04 dist-test-slave-tc2s krb5kdc[9512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:39:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:39:04Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.24.76.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.24.76.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.24.76.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/rangeradmin_127.24.76.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.24.76.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/rangeradmin_127.24.76.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.24.76.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.24.76.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.24.76.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/rangerlookup_127.24.76.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.24.76.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/rangerlookup_127.24.76.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.76.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.24.76.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.76.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/HTTP_127.24.76.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.76.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/HTTP_127.24.76.84@KRBTEST.COM.keytab.
I20260808 11:39:04.747035 24881 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskryQpdG/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/postgres -l logfile start
2026-08-08 11:39:07.863 UTC [9557] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 11:39:07.864 UTC [9557] LOG: listening on IPv4 address "127.24.76.84", port 38983
2026-08-08 11:39:07.867 UTC [9557] LOG: listening on Unix socket "/tmp/.s.PGSQL.38983"
2026-08-08 11:39:07.888 UTC [9565] LOG: database system was shut down at 2026-08-08 11:39:06 UTC
2026-08-08 11:39:07.892 UTC [9557] LOG: database system is ready to accept connections
I20260808 11:39:08.965243 24881 mini_postgres.cc:96] Postgres bound to 38983
2026-08-08 11:39:08.990 UTC [9578] FATAL: database "slave" does not exist
127.24.76.84:38983 - accepting connections
I20260808 11:39:08.991638 24881 mini_ranger.cc:161] Starting Ranger...
I20260808 11:39:09.021881 24881 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 11:39:09.439157 24881 mini_ranger.cc:87] Created ranger Postgres database
I20260808 11:39:09.439293 24881 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ranger-admin
2026-08-08 11:39:09,778 [I] DB FLAVOR :POSTGRES
2026-08-08 11:39:09,778 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 11:39:09,778 [I] Checking connection..
2026-08-08 11:39:09,779 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 11:39:10,026 [I] Checking connection passed.
2026-08-08 11:39:10,027 [I] --------- Verifying version history table ---------
2026-08-08 11:39:10,027 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 11:39:10,263 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 11:39:10,264 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 11:39:10,514 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 11:39:10,514 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 11:39:10,514 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 11:39:10.761 UTC [9671] WARNING: there is no transaction in progress
2026-08-08 11:39:10,767 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 11:39:10,768 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 11:39:10,997 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 11:39:10,997 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 11:39:10,997 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 11:39:11,241 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 11:39:11,485 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-tc2s.c.gcp-upstream.internal','N') ;"
2026-08-08 11:39:11,715 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 11:39:11,956 [I] Table x_portal_user does not exist in database ranger
2026-08-08 11:39:11,956 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 11:39:12,181 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 11:39:12,182 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 11:39:12,429 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 11:39:12,660 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 11:39:12,660 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 11:39:13.538 UTC [9832] WARNING: there is no transaction in progress
2026-08-08 11:39:13.552 UTC [9832] WARNING: there is no transaction in progress
2026-08-08 11:39:13.566 UTC [9832] WARNING: there is no transaction in progress
2026-08-08 11:39:13.580 UTC [9832] WARNING: there is no transaction in progress
2026-08-08 11:39:13.594 UTC [9832] WARNING: there is no transaction in progress
2026-08-08 11:39:13.613 UTC [9832] WARNING: there is no transaction in progress
2026-08-08 11:39:13.663 UTC [9832] WARNING: there is no transaction in progress
2026-08-08 11:39:13.669 UTC [9832] WARNING: there is no transaction in progress
2026-08-08 11:39:13.682 UTC [9832] WARNING: there is no transaction in progress
2026-08-08 11:39:13.691 UTC [9832] WARNING: there is no transaction in progress
2026-08-08 11:39:13.700 UTC [9832] WARNING: there is no transaction in progress
2026-08-08 11:39:13.713 UTC [9832] WARNING: there is no transaction in progress
2026-08-08 11:39:13.721 UTC [9832] WARNING: there is no transaction in progress
2026-08-08 11:39:13.727 UTC [9832] WARNING: there is no transaction in progress
2026-08-08 11:39:13.733 UTC [9832] WARNING: there is no transaction in progress
2026-08-08 11:39:13.739 UTC [9832] WARNING: there is no transaction in progress
2026-08-08 11:39:14,283 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 11:39:14,283 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 11:39:14,514 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 11:39:14,514 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 11:39:14,745 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 11:39:14,745 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 11:39:15,001 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 11:39:15,002 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 11:39:15,236 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 11:39:15,482 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-tc2s.c.gcp-upstream.internal';"
2026-08-08 11:39:15,728 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 11:39:15,728 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:38983/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 11:39:15,990 [I] DB_PATCHES have already been applied
I20260808 11:39:15.996707 24881 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ranger-admin:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskryQpdG/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 11:39:15.996811 24881 mini_ranger.cc:193] Using host: 127.24.76.84
I20260808 11:39:16.021801 24881 mini_ranger.cc:227] Ranger admin URL: http://127.24.76.84:49563
Aug 08, 2026 11:39:16 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 11:39:16 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 11:39:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 11:39:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 11:39:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 11:39:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 11:39:16 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 11:39:17.235443 24881 mini_ranger.cc:160] Time spent starting Ranger: real 8.244s user 0.007s sys 0.055s
Aug 08, 2026 11:39:36 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 11:39:36 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 11:39:36 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 11:39:36 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 11:39:36 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 11:39:36 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 11:39:36 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 11:39:36 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 11:39:36 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 11:39:36 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 11:39:36 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 11:39:36 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 11:39:36 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 11:39:36 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 11:39:36 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 11:39:36 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 11:39:36 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 11:39:37 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 11:39:38.229305 24881 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.24.76.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.24.76.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.24.76.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/rangerkms_127.24.76.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.24.76.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/rangerkms_127.24.76.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.76.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.24.76.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.76.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/HTTP_127.24.76.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.76.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/HTTP_127.24.76.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 11:39:38 dist-test-slave-tc2s krb5kdc[9512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189178, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 11:39:38.418367 24881 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 11:39:38.440925 24881 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 11:39:39.138635 24881 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 11:39:39.138794 24881 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ranger-kms
I20260808 11:39:39.138900 24881 mini_ranger_kms.cc:227] Using postgres at 127.24.76.84:38983
2026-08-08 11:39:40,012 [I] DB FLAVOR :POSTGRES
2026-08-08 11:39:40,013 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 11:39:40,013 [I] Checking connection
2026-08-08 11:39:40,013 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:38983/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 11:39:40,268 [I] connection success
2026-08-08 11:39:40,268 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 11:39:40,269 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 11:39:40,269 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:38983/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 11:39:40,510 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 11:39:40,510 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 11:39:40,510 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 11:39:40,511 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:38983/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 11:39:40,770 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 11:39:41.203542 24881 mini_ranger_kms.cc:310] Created kms service
I20260808 11:39:41.408526 24881 mini_ranger_kms.cc:326] Created kudu user
I20260808 11:39:41.449443 24881 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 11:39:41 dist-test-slave-tc2s krb5kdc[9512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189178, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.24.76.84@KRBTEST.COM
I20260808 11:39:41.800542 24881 mini_ranger_kms.cc:384] Added ranger policy
I20260808 11:39:41.801779 24881 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ranger-kms:/tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 11:39:41.801846 24881 mini_ranger_kms.cc:240] Using host: 127.24.76.84
I20260808 11:39:41.835127 24881 mini_ranger_kms.cc:276] Ranger KMS PID: 10107
I20260808 11:39:41.835295 24881 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.24.76.84:50355
Aug 08, 2026 11:39:42 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 11:39:42 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 11:39:43 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 11:39:43 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 11:39:43 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 11:39:43 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 11:39:44 dist-test-slave-tc2s krb5kdc[9512](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786189184, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.76.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:39:44.409433 24881 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.991s user 0.094s sys 0.475s
I20260808 11:39:44.409817 24881 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 11:39:44.410229 24881 mini_ranger_kms.cc:401] 127.24.76.84:50355/kms/v1/keys
Aug 08 11:39:44 dist-test-slave-tc2s krb5kdc[9512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189184, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.76.84@KRBTEST.COM for HTTP/127.24.76.84@KRBTEST.COM
Aug 08 11:39:45 dist-test-slave-tc2s krb5kdc[9512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189184, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.76.84@KRBTEST.COM for HTTP/127.24.76.84@KRBTEST.COM
Aug 08 11:39:47 dist-test-slave-tc2s krb5kdc[9512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.24.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
I20260808 11:39:47.710374 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:40965
--webserver_interface=127.24.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:45611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:40965
--ranger_config_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.76.84:50355/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:39:48.163003 10176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:39:48.163313 10176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:39:48.163451 10176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:39:48.171972 10176 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:39:48.172135 10176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:39:48.172217 10176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:39:48.172360 10176 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:39:48.172468 10176 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:39:48.183593 10176 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:45611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/master-0/wal
--ranger_kms_url=127.24.76.84:50355/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:40965
--ranger_config_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:40965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:39:48.184867 10176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:39:48.186192 10176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:39:48.199617 10188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:39:48.199671 10183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:39:48.199858 10182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:39:48.200510 10176 server_base.cc:1034] running on GCE node
I20260808 11:39:48.201247 10176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:39:48.202754 10176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:39:48.204008 10176 hybrid_clock.cc:648] HybridClock initialized: now 1786189188203915 us; error 34 us; skew 500 ppm
I20260808 11:39:48.204386 10176 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:39:48 dist-test-slave-tc2s krb5kdc[9512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189188, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:39:48.207903 10176 init.cc:377] Logged in from keytab as kudu/127.24.76.126@KRBTEST.COM (short username kudu)
I20260808 11:39:48.209486 10176 webserver.cc:460] Webserver started at http://127.24.76.126:44107/ using document root <none> and password file <none>
I20260808 11:39:48.210211 10176 fs_manager.cc:359] Metadata directory not provided
I20260808 11:39:48.210388 10176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:39:48.210700 10176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 11:39:48 dist-test-slave-tc2s krb5kdc[9512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189188, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for HTTP/127.24.76.84@KRBTEST.COM
I20260808 11:39:48.270924 10176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "bf8d8c21477f401684f328da22959a54"
format_stamp: "Formatted at 2026-08-08 11:39:48 on dist-test-slave-tc2s"
server_key: "0b21ff212604d20fe65c66a2e56ae98d"
server_key_iv: "51421570c51970319c3d9eee3c7aa2ba"
server_key_version: "kuduclusterkey@0"
I20260808 11:39:48.271834 10176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "bf8d8c21477f401684f328da22959a54"
format_stamp: "Formatted at 2026-08-08 11:39:48 on dist-test-slave-tc2s"
server_key: "0b21ff212604d20fe65c66a2e56ae98d"
server_key_iv: "51421570c51970319c3d9eee3c7aa2ba"
server_key_version: "kuduclusterkey@0"
I20260808 11:39:48.278101 10176 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260808 11:39:48.293931 10203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:39:48.295397 10176 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 11:39:48.295571 10176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "bf8d8c21477f401684f328da22959a54"
format_stamp: "Formatted at 2026-08-08 11:39:48 on dist-test-slave-tc2s"
server_key: "0b21ff212604d20fe65c66a2e56ae98d"
server_key_iv: "51421570c51970319c3d9eee3c7aa2ba"
server_key_version: "kuduclusterkey@0"
I20260808 11:39:48.295773 10176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:39:48.309850 10176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:39:48.313714 10176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:39:48.314178 10176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:39:48.336309 10176 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:40965
I20260808 11:39:48.336334 10280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:40965 every 8 connection(s)
I20260808 11:39:48.337620 10176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:39:48.338882 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 10176
I20260808 11:39:48.339112 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:39:48.340090 24881 external_mini_cluster.cc:1428] Setting key 210bd50b0c2ef825cc764c88cf40c3a7
I20260808 11:39:48.342610 10281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 11:39:48 dist-test-slave-tc2s krb5kdc[9512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:39:48.356952 10281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf8d8c21477f401684f328da22959a54: Bootstrap starting.
I20260808 11:39:48.360965 10281 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf8d8c21477f401684f328da22959a54: Neither blocks nor log segments found. Creating new log.
I20260808 11:39:48.362277 10281 log.cc:824] T 00000000000000000000000000000000 P bf8d8c21477f401684f328da22959a54: Log is configured to *not* fsync() on all Append() calls
I20260808 11:39:48.363790 10284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:48.342361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40400 (local address 127.24.76.126:40965)
0808 11:39:48.343387 (+ 1026us) server_negotiation.cc:206] Beginning negotiation
0808 11:39:48.343399 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 11:39:48.343440 (+ 41us) server_negotiation.cc:407] Connection header received
0808 11:39:48.345611 (+ 2171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:39:48.345633 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:39:48.346034 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:39:48.346409 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:39:48.347726 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:48.349313 (+ 1587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:48.350555 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:48.351453 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:48.351618 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:48.355402 (+ 3784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:39:48.355412 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:39:48.355425 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:39:48.355493 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:39:48.359325 (+ 3832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:39:48.359964 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:39:48.359970 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:39:48.359976 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:39:48.360168 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:39:48.360547 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:39:48.360551 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:39:48.360553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:39:48.361136 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:39:48.361536 (+ 400us) server_negotiation.cc:1035] Waiting for connection context
0808 11:39:48.361930 (+ 394us) server_negotiation.cc:299] Negotiation successful
0808 11:39:48.362279 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":811,"thread_start_us":579,"threads_started":1}
I20260808 11:39:48.365552 10281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf8d8c21477f401684f328da22959a54: No bootstrap required, opened a new log
I20260808 11:39:48.374310 10281 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf8d8c21477f401684f328da22959a54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf8d8c21477f401684f328da22959a54" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 40965 } }
I20260808 11:39:48.374680 10281 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf8d8c21477f401684f328da22959a54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:39:48.374835 10281 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf8d8c21477f401684f328da22959a54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf8d8c21477f401684f328da22959a54, State: Initialized, Role: FOLLOWER
I20260808 11:39:48.375473 10281 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf8d8c21477f401684f328da22959a54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf8d8c21477f401684f328da22959a54" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 40965 } }
I20260808 11:39:48.375804 10281 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf8d8c21477f401684f328da22959a54 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:39:48.375993 10281 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf8d8c21477f401684f328da22959a54 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:39:48.376214 10281 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf8d8c21477f401684f328da22959a54 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:39:48.378679 10281 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf8d8c21477f401684f328da22959a54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf8d8c21477f401684f328da22959a54" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 40965 } }
I20260808 11:39:48.379163 10281 leader_election.cc:302] T 00000000000000000000000000000000 P bf8d8c21477f401684f328da22959a54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf8d8c21477f401684f328da22959a54; no voters:
I20260808 11:39:48.379912 10281 leader_election.cc:288] T 00000000000000000000000000000000 P bf8d8c21477f401684f328da22959a54 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:39:48.380084 10290 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf8d8c21477f401684f328da22959a54 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:39:48.381142 10290 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf8d8c21477f401684f328da22959a54 [term 1 LEADER]: Becoming Leader. State: Replica: bf8d8c21477f401684f328da22959a54, State: Running, Role: LEADER
I20260808 11:39:48.381768 10290 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf8d8c21477f401684f328da22959a54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf8d8c21477f401684f328da22959a54" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 40965 } }
I20260808 11:39:48.382501 10281 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf8d8c21477f401684f328da22959a54 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:39:48.385599 10292 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf8d8c21477f401684f328da22959a54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf8d8c21477f401684f328da22959a54. Latest consensus state: current_term: 1 leader_uuid: "bf8d8c21477f401684f328da22959a54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf8d8c21477f401684f328da22959a54" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 40965 } } }
I20260808 11:39:48.386456 10292 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf8d8c21477f401684f328da22959a54 [sys.catalog]: This master's current role is: LEADER
I20260808 11:39:48.385560 10291 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf8d8c21477f401684f328da22959a54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf8d8c21477f401684f328da22959a54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf8d8c21477f401684f328da22959a54" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 40965 } } }
I20260808 11:39:48.387328 10291 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf8d8c21477f401684f328da22959a54 [sys.catalog]: This master's current role is: LEADER
I20260808 11:39:48.396368 10304 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:39:48.397159 10281 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 11:39:48.412662 10304 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:39:48.424504 10304 catalog_manager.cc:1347] Generated new cluster ID: 2e8dc7d83d5c4373b280f1e67cf8b646
I20260808 11:39:48.424675 10304 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:39:48.460733 10304 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:39:48.462780 10304 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:39:48.487908 10304 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf8d8c21477f401684f328da22959a54: Generated new TSK 0
I20260808 11:39:48.488945 10304 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab.
Entry for principal kudu/127.24.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab.
I20260808 11:39:49.257561 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.76.84:50355/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.65:0
--local_ip_for_outbound_sockets=127.24.76.65
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:40965
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:45611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 11:39:49 dist-test-slave-tc2s krb5kdc[9512](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786189189, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 11:39:49.746126 10348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:39:49.746587 10348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:39:49.746791 10348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:39:49.758505 10348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:39:49.758728 10348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:39:49.758967 10348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.65
I20260808 11:39:49.775897 10348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:45611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.24.76.84:50355/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:40965
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.65
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:39:49.777329 10348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:39:49.778715 10348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:39:49.795567 10364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:39:49.795874 10365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:39:49.796842 10367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:39:49.797561 10348 server_base.cc:1034] running on GCE node
I20260808 11:39:49.798254 10348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:39:49.799401 10348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:39:49.800690 10348 hybrid_clock.cc:648] HybridClock initialized: now 1786189189800633 us; error 52 us; skew 500 ppm
I20260808 11:39:49.801012 10348 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:39:49 dist-test-slave-tc2s krb5kdc[9512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189189, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:39:49.804719 10348 init.cc:377] Logged in from keytab as kudu/127.24.76.65@KRBTEST.COM (short username kudu)
I20260808 11:39:49.806835 10348 webserver.cc:460] Webserver started at http://127.24.76.65:43755/ using document root <none> and password file <none>
I20260808 11:39:49.807844 10348 fs_manager.cc:359] Metadata directory not provided
I20260808 11:39:49.807997 10348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:39:49.808535 10348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 11:39:49 dist-test-slave-tc2s krb5kdc[9512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189189, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for HTTP/127.24.76.84@KRBTEST.COM
I20260808 11:39:49.836025 10348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-0/data/instance:
uuid: "f36c198c12954896936a8d42700771a2"
format_stamp: "Formatted at 2026-08-08 11:39:49 on dist-test-slave-tc2s"
server_key: "8d0a76e97fa61d6ee9d8312fcf65c723"
server_key_iv: "0c9787d78454fa7cee2b4e36f47582e4"
server_key_version: "kuduclusterkey@0"
I20260808 11:39:49.837252 10348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance:
uuid: "f36c198c12954896936a8d42700771a2"
format_stamp: "Formatted at 2026-08-08 11:39:49 on dist-test-slave-tc2s"
server_key: "8d0a76e97fa61d6ee9d8312fcf65c723"
server_key_iv: "0c9787d78454fa7cee2b4e36f47582e4"
server_key_version: "kuduclusterkey@0"
I20260808 11:39:49.854537 10348 fs_manager.cc:691] Time spent creating directory manager: real 0.015s user 0.009s sys 0.004s
I20260808 11:39:49.883280 10381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:39:49.885293 10348 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.008s sys 0.000s
I20260808 11:39:49.885699 10348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-0/wal
uuid: "f36c198c12954896936a8d42700771a2"
format_stamp: "Formatted at 2026-08-08 11:39:49 on dist-test-slave-tc2s"
server_key: "8d0a76e97fa61d6ee9d8312fcf65c723"
server_key_iv: "0c9787d78454fa7cee2b4e36f47582e4"
server_key_version: "kuduclusterkey@0"
I20260808 11:39:49.886132 10348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:39:49.922075 10348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:39:49.928586 10348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:39:49.929075 10348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:39:49.930792 10348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:39:49.932655 10348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:39:49.932744 10348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:39:49.932862 10348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:39:49.932926 10348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 11:39:49 dist-test-slave-tc2s krb5kdc[9512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189189, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for HTTP/127.24.76.84@KRBTEST.COM
I20260808 11:39:50.342631 10348 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.65:43385
I20260808 11:39:50.344661 10348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
I20260808 11:39:50.344910 10540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.65:43385 every 8 connection(s)
I20260808 11:39:50.353194 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 10348
I20260808 11:39:50.353427 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance
I20260808 11:39:50.354349 24881 external_mini_cluster.cc:1428] Setting key a7205cc3558c3744c3f21b05e54fed09
Aug 08 11:39:50 dist-test-slave-tc2s krb5kdc[9512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189189, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:39:50.389436 10543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:50.351138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:45797 (local address 127.24.76.126:40965)
0808 11:39:50.367268 (+ 16130us) server_negotiation.cc:206] Beginning negotiation
0808 11:39:50.367274 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:39:50.367299 (+ 25us) server_negotiation.cc:407] Connection header received
0808 11:39:50.369660 (+ 2361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:39:50.369666 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:39:50.369754 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:39:50.369868 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:39:50.372356 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:50.373656 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:50.376385 (+ 2729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:50.377171 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:50.377270 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:50.382067 (+ 4797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:39:50.382076 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:39:50.382081 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:39:50.382138 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:39:50.385585 (+ 3447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:39:50.386513 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:39:50.386520 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:39:50.386523 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:39:50.386718 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:39:50.387284 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:39:50.387289 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:39:50.387292 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:39:50.387675 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:39:50.387796 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 11:39:50.388866 (+ 1070us) server_negotiation.cc:299] Negotiation successful
0808 11:39:50.389072 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15994,"thread_start_us":17720,"threads_started":1}
I20260808 11:39:50.392143 10544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:50.351989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:40965 (local address 127.24.76.65:45797)
0808 11:39:50.364615 (+ 12626us) negotiation.cc:107] Waiting for socket to connect
0808 11:39:50.364679 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 11:39:50.369197 (+ 4518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:39:50.370383 (+ 1186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:39:50.370395 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:39:50.371110 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:39:50.372138 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:39:50.372164 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:39:50.373869 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:39:50.373874 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:39:50.376197 (+ 2323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:39:50.376215 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:39:50.377689 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:39:50.377698 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:39:50.378016 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:50.378827 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:39:50.378858 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:39:50.381740 (+ 2882us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:39:50.385744 (+ 4004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:39:50.385753 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:39:50.385771 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:39:50.386382 (+ 611us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:39:50.386861 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:39:50.386866 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:39:50.386870 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:39:50.387162 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:39:50.387895 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:39:50.387902 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:39:50.388442 (+ 540us) client_negotiation.cc:769] Sending connection context
0808 11:39:50.389939 (+ 1497us) client_negotiation.cc:240] Negotiation successful
0808 11:39:50.390439 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12279,"thread_start_us":13792,"threads_started":1}
I20260808 11:39:50.394563 10541 heartbeater.cc:347] Connected to a master server at 127.24.76.126:40965
I20260808 11:39:50.394963 10541 heartbeater.cc:464] Registering TS with master...
I20260808 11:39:50.396054 10541 heartbeater.cc:511] Master 127.24.76.126:40965 requested a full tablet report, sending...
I20260808 11:39:50.399130 10224 ts_manager.cc:194] Registered new tserver with Master: f36c198c12954896936a8d42700771a2 (127.24.76.65:43385)
I20260808 11:39:50.401535 10224 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.65@KRBTEST.COM'} at 127.24.76.65:45797
WARNING: no policy specified for kudu/127.24.76.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.66@KRBTEST.COM" created.
Aug 08 11:39:50 dist-test-slave-tc2s krb5kdc[9512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189189, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for HTTP/127.24.76.84@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab.
Entry for principal kudu/127.24.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab.
I20260808 11:39:50.455528 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.76.84:50355/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.66:0
--local_ip_for_outbound_sockets=127.24.76.66
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:40965
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:45611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:39:51.009120 10554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:39:51.009389 10554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:39:51.009516 10554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:39:51.016821 10554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:39:51.016927 10554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:39:51.017077 10554 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.66
I20260808 11:39:51.027345 10554 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:45611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.24.76.84:50355/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:40965
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.66
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:39:51.028656 10554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:39:51.030109 10554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:39:51.046926 10577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:39:51.047173 10576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:39:51.047003 10579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:39:51.048640 10554 server_base.cc:1034] running on GCE node
I20260808 11:39:51.049386 10554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:39:51.050733 10554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:39:51.052011 10554 hybrid_clock.cc:648] HybridClock initialized: now 1786189191051936 us; error 54 us; skew 500 ppm
I20260808 11:39:51.052364 10554 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:39:51 dist-test-slave-tc2s krb5kdc[9512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189191, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:39:51.056396 10554 init.cc:377] Logged in from keytab as kudu/127.24.76.66@KRBTEST.COM (short username kudu)
I20260808 11:39:51.058126 10554 webserver.cc:460] Webserver started at http://127.24.76.66:46647/ using document root <none> and password file <none>
I20260808 11:39:51.058841 10554 fs_manager.cc:359] Metadata directory not provided
I20260808 11:39:51.058943 10554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:39:51.059247 10554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 11:39:51 dist-test-slave-tc2s krb5kdc[9512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189191, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for HTTP/127.24.76.84@KRBTEST.COM
I20260808 11:39:51.077654 10554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-1/data/instance:
uuid: "3ef52967ad864360a296dfc9effc0b05"
format_stamp: "Formatted at 2026-08-08 11:39:51 on dist-test-slave-tc2s"
server_key: "8efd066dd0cd697fb0f4aec14ea6df1c"
server_key_iv: "a201cfcc368330e313f333e8f74c5ccd"
server_key_version: "kuduclusterkey@0"
I20260808 11:39:51.078774 10554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance:
uuid: "3ef52967ad864360a296dfc9effc0b05"
format_stamp: "Formatted at 2026-08-08 11:39:51 on dist-test-slave-tc2s"
server_key: "8efd066dd0cd697fb0f4aec14ea6df1c"
server_key_iv: "a201cfcc368330e313f333e8f74c5ccd"
server_key_version: "kuduclusterkey@0"
I20260808 11:39:51.086017 10554 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.002s
I20260808 11:39:51.101363 10586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:39:51.102829 10554 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 11:39:51.102998 10554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-1/wal
uuid: "3ef52967ad864360a296dfc9effc0b05"
format_stamp: "Formatted at 2026-08-08 11:39:51 on dist-test-slave-tc2s"
server_key: "8efd066dd0cd697fb0f4aec14ea6df1c"
server_key_iv: "a201cfcc368330e313f333e8f74c5ccd"
server_key_version: "kuduclusterkey@0"
I20260808 11:39:51.103183 10554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:39:51.127322 10554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:39:51.131420 10554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:39:51.131726 10554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:39:51.132730 10554 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:39:51.134371 10554 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:39:51.134450 10554 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:39:51.134543 10554 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:39:51.134590 10554 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:39:51.184674 10554 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.66:42815
I20260808 11:39:51.184712 10733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.66:42815 every 8 connection(s)
I20260808 11:39:51.186157 10554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
I20260808 11:39:51.193743 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 10554
I20260808 11:39:51.193913 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance
I20260808 11:39:51.194348 24881 external_mini_cluster.cc:1428] Setting key a4d72c47fae743559ade84eb648cf536
Aug 08 11:39:51 dist-test-slave-tc2s krb5kdc[9512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189191, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:39:51.212337 10739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:51.189105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:37849 (local address 127.24.76.126:40965)
0808 11:39:51.189694 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0808 11:39:51.189699 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:39:51.190870 (+ 1171us) server_negotiation.cc:407] Connection header received
0808 11:39:51.193046 (+ 2176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:39:51.193052 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:39:51.193135 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:39:51.193232 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:39:51.195138 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:51.196075 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:51.197657 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:51.198103 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:51.198214 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:51.206049 (+ 7835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:39:51.206055 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:39:51.206058 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:39:51.206089 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:39:51.209052 (+ 2963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:39:51.209778 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:39:51.209782 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:39:51.209785 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:39:51.209940 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:39:51.210441 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:39:51.210451 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:39:51.210454 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:39:51.210713 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:39:51.210890 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 11:39:51.211620 (+ 730us) server_negotiation.cc:299] Negotiation successful
0808 11:39:51.211844 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":276,"threads_started":1}
I20260808 11:39:51.213194 10742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:51.189816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:40965 (local address 127.24.76.66:37849)
0808 11:39:51.190714 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0808 11:39:51.190752 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 11:39:51.192790 (+ 2038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:39:51.193475 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:39:51.193485 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:39:51.194043 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:39:51.194721 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:39:51.194742 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:39:51.196225 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:39:51.196229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:39:51.197521 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:39:51.197534 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:39:51.198320 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:39:51.198327 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:39:51.198646 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:51.203212 (+ 4566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:39:51.203235 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:39:51.205813 (+ 2578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:39:51.209181 (+ 3368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:39:51.209190 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:39:51.209200 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:39:51.209657 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:39:51.210042 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:39:51.210047 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:39:51.210050 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:39:51.210321 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:39:51.210844 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:39:51.210852 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:39:51.211157 (+ 305us) client_negotiation.cc:769] Sending connection context
0808 11:39:51.211415 (+ 258us) client_negotiation.cc:240] Negotiation successful
0808 11:39:51.211936 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":416,"threads_started":1}
I20260808 11:39:51.214892 10735 heartbeater.cc:347] Connected to a master server at 127.24.76.126:40965
I20260808 11:39:51.215240 10735 heartbeater.cc:464] Registering TS with master...
I20260808 11:39:51.215970 10735 heartbeater.cc:511] Master 127.24.76.126:40965 requested a full tablet report, sending...
I20260808 11:39:51.217443 10224 ts_manager.cc:194] Registered new tserver with Master: 3ef52967ad864360a296dfc9effc0b05 (127.24.76.66:42815)
I20260808 11:39:51.218264 10224 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.66@KRBTEST.COM'} at 127.24.76.66:37849
WARNING: no policy specified for kudu/127.24.76.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab.
Entry for principal kudu/127.24.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab.
I20260808 11:39:51.255224 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.76.84:50355/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.67:0
--local_ip_for_outbound_sockets=127.24.76.67
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:40965
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:45611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:39:51.404899 10541 heartbeater.cc:503] Master 127.24.76.126:40965 was elected leader, sending a full tablet report...
W20260808 11:39:51.701683 10746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:39:51.702033 10746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:39:51.702248 10746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:39:51.711879 10746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:39:51.712073 10746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:39:51.712297 10746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.67
I20260808 11:39:51.723785 10746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:45611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.24.76.84:50355/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:40965
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.67
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:39:51.725165 10746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:39:51.726495 10746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:39:51.741959 10751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:39:51.741864 10754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:39:51.741925 10752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:39:51.742848 10746 server_base.cc:1034] running on GCE node
I20260808 11:39:51.743405 10746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:39:51.744308 10746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:39:51.745553 10746 hybrid_clock.cc:648] HybridClock initialized: now 1786189191745462 us; error 40 us; skew 500 ppm
I20260808 11:39:51.745875 10746 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:39:51 dist-test-slave-tc2s krb5kdc[9512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189191, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:39:51.749214 10746 init.cc:377] Logged in from keytab as kudu/127.24.76.67@KRBTEST.COM (short username kudu)
I20260808 11:39:51.750689 10746 webserver.cc:460] Webserver started at http://127.24.76.67:45691/ using document root <none> and password file <none>
I20260808 11:39:51.751410 10746 fs_manager.cc:359] Metadata directory not provided
I20260808 11:39:51.751576 10746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:39:51.751955 10746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 11:39:51 dist-test-slave-tc2s krb5kdc[9512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189191, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for HTTP/127.24.76.84@KRBTEST.COM
I20260808 11:39:51.769034 10746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-2/data/instance:
uuid: "0864cd8a9da0422a9dc422a2dc9c5666"
format_stamp: "Formatted at 2026-08-08 11:39:51 on dist-test-slave-tc2s"
server_key: "f724669d61b87563b3dc0733657a890c"
server_key_iv: "190330aed24236cdaf330b34c24db568"
server_key_version: "kuduclusterkey@0"
I20260808 11:39:51.770251 10746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance:
uuid: "0864cd8a9da0422a9dc422a2dc9c5666"
format_stamp: "Formatted at 2026-08-08 11:39:51 on dist-test-slave-tc2s"
server_key: "f724669d61b87563b3dc0733657a890c"
server_key_iv: "190330aed24236cdaf330b34c24db568"
server_key_version: "kuduclusterkey@0"
I20260808 11:39:51.776417 10746 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260808 11:39:51.790897 10761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:39:51.792337 10746 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:39:51.792502 10746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-2/wal
uuid: "0864cd8a9da0422a9dc422a2dc9c5666"
format_stamp: "Formatted at 2026-08-08 11:39:51 on dist-test-slave-tc2s"
server_key: "f724669d61b87563b3dc0733657a890c"
server_key_iv: "190330aed24236cdaf330b34c24db568"
server_key_version: "kuduclusterkey@0"
I20260808 11:39:51.792711 10746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:39:51.806201 10746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:39:51.810175 10746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:39:51.810459 10746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:39:51.811380 10746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:39:51.812870 10746 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:39:51.812940 10746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:39:51.813057 10746 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:39:51.813105 10746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:39:51.864151 10746 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.67:45943
I20260808 11:39:51.864248 10928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.67:45943 every 8 connection(s)
I20260808 11:39:51.865497 10746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
I20260808 11:39:51.871592 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 10746
I20260808 11:39:51.871745 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance
I20260808 11:39:51.872215 24881 external_mini_cluster.cc:1428] Setting key dd0e4cb74b925f4999f62d194f50a326
Aug 08 11:39:51 dist-test-slave-tc2s krb5kdc[9512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189191, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:39:51.887511 10934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:51.868515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:55097 (local address 127.24.76.126:40965)
0808 11:39:51.869101 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0808 11:39:51.869106 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:39:51.870134 (+ 1028us) server_negotiation.cc:407] Connection header received
0808 11:39:51.872455 (+ 2321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:39:51.872461 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:39:51.872545 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:39:51.872644 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:39:51.874682 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:51.875590 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:51.877283 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:51.877735 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:51.877844 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:51.881244 (+ 3400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:39:51.881249 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:39:51.881252 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:39:51.881288 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:39:51.884295 (+ 3007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:39:51.885011 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:39:51.885015 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:39:51.885017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:39:51.885146 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:39:51.885645 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:39:51.885649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:39:51.885651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:39:51.885977 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:39:51.886149 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 11:39:51.886756 (+ 607us) server_negotiation.cc:299] Negotiation successful
0808 11:39:51.887015 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":279,"threads_started":1}
I20260808 11:39:51.888427 10935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:51.869065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:40965 (local address 127.24.76.67:55097)
0808 11:39:51.869987 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0808 11:39:51.870026 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 11:39:51.872189 (+ 2163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:39:51.872937 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:39:51.872948 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:39:51.873676 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:39:51.874526 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:39:51.874547 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:39:51.875757 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:39:51.875761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:39:51.877108 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:39:51.877128 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:39:51.877975 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:39:51.877984 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:39:51.878278 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:51.878903 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:39:51.878925 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:39:51.881042 (+ 2117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:39:51.884420 (+ 3378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:39:51.884429 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:39:51.884451 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:39:51.884888 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:39:51.885250 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:39:51.885254 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:39:51.885258 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:39:51.885527 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:39:51.886097 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:39:51.886106 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:39:51.886404 (+ 298us) client_negotiation.cc:769] Sending connection context
0808 11:39:51.886636 (+ 232us) client_negotiation.cc:240] Negotiation successful
0808 11:39:51.887138 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"thread_start_us":479,"threads_started":1}
I20260808 11:39:51.889662 10931 heartbeater.cc:347] Connected to a master server at 127.24.76.126:40965
I20260808 11:39:51.889977 10931 heartbeater.cc:464] Registering TS with master...
I20260808 11:39:51.890774 10931 heartbeater.cc:511] Master 127.24.76.126:40965 requested a full tablet report, sending...
I20260808 11:39:51.892283 10224 ts_manager.cc:194] Registered new tserver with Master: 0864cd8a9da0422a9dc422a2dc9c5666 (127.24.76.67:45943)
I20260808 11:39:51.893112 10224 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.67@KRBTEST.COM'} at 127.24.76.67:55097
I20260808 11:39:51.899578 24881 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:39:51.918222 10934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:51.904586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40410 (local address 127.24.76.126:40965)
0808 11:39:51.904722 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 11:39:51.904726 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:39:51.905252 (+ 526us) server_negotiation.cc:407] Connection header received
0808 11:39:51.905443 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:39:51.905455 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:39:51.905520 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:39:51.905592 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:39:51.907575 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:51.908414 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:51.909782 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:51.910326 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:51.910415 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:51.912973 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:39:51.912977 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:39:51.912980 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:39:51.913007 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:39:51.915922 (+ 2915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:39:51.916503 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:39:51.916506 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:39:51.916509 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:39:51.916657 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:39:51.916996 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:39:51.916999 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:39:51.917001 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:39:51.917269 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:39:51.917504 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0808 11:39:51.917823 (+ 319us) server_negotiation.cc:299] Negotiation successful
0808 11:39:51.918014 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 11:39:51.921952 10224 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40410:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:39:51.930019 10224 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:39:51.965974 10947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:51.942031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:45943 (local address 127.0.0.1:39372)
0808 11:39:51.952000 (+ 9969us) negotiation.cc:107] Waiting for socket to connect
0808 11:39:51.952045 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 11:39:51.953257 (+ 1212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:39:51.956502 (+ 3245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:39:51.956508 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:39:51.956536 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:39:51.957042 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:39:51.957059 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:39:51.959330 (+ 2271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:39:51.959335 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:39:51.962191 (+ 2856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:39:51.962204 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:39:51.964850 (+ 2646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:39:51.964859 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:39:51.965097 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:51.965163 (+ 66us) client_negotiation.cc:769] Sending connection context
0808 11:39:51.965299 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 11:39:51.965510 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9684,"thread_start_us":16328,"threads_started":1}
I20260808 11:39:51.966374 10948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:51.942499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39372 (local address 127.24.76.67:45943)
0808 11:39:51.955883 (+ 13384us) server_negotiation.cc:206] Beginning negotiation
0808 11:39:51.955891 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:39:51.955926 (+ 35us) server_negotiation.cc:407] Connection header received
0808 11:39:51.956039 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:39:51.956047 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:39:51.956297 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:39:51.956491 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:39:51.957292 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:51.959137 (+ 1845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:51.962375 (+ 3238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:51.964256 (+ 1881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:51.964365 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:51.964598 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0808 11:39:51.965719 (+ 1121us) server_negotiation.cc:299] Negotiation successful
0808 11:39:51.965938 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13214,"thread_start_us":13046,"threads_started":1}
I20260808 11:39:51.967476 10950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:51.942585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:43385 (local address 127.0.0.1:46156)
0808 11:39:51.958049 (+ 15464us) negotiation.cc:107] Waiting for socket to connect
0808 11:39:51.958071 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 11:39:51.958322 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:39:51.959464 (+ 1142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:39:51.959468 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:39:51.959494 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:39:51.960383 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:39:51.960395 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:39:51.962509 (+ 2114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:39:51.962514 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:39:51.964504 (+ 1990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:39:51.964517 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:39:51.966604 (+ 2087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:39:51.966613 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:39:51.966808 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:51.966839 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 11:39:51.966942 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 11:39:51.967083 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15264,"thread_start_us":8909,"threads_started":1}
I20260808 11:39:51.967706 10949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:51.942731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46156 (local address 127.24.76.65:43385)
0808 11:39:51.955065 (+ 12334us) server_negotiation.cc:206] Beginning negotiation
0808 11:39:51.955076 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:39:51.958185 (+ 3109us) server_negotiation.cc:407] Connection header received
0808 11:39:51.958617 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:39:51.958624 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:39:51.958885 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:39:51.959110 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:39:51.960580 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:51.962346 (+ 1766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:51.964652 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:51.966059 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:51.966191 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:51.966416 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 11:39:51.967004 (+ 588us) server_negotiation.cc:299] Negotiation successful
0808 11:39:51.967167 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12130,"thread_start_us":12332,"threads_started":1}
I20260808 11:39:51.969419 10946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:51.944284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:42815 (local address 127.0.0.1:49858)
0808 11:39:51.958750 (+ 14466us) negotiation.cc:107] Waiting for socket to connect
0808 11:39:51.958767 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:39:51.958885 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:39:51.959719 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:39:51.959723 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:39:51.959753 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:39:51.960717 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:39:51.960732 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:39:51.962802 (+ 2070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:39:51.962807 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:39:51.965487 (+ 2680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:39:51.965502 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:39:51.968523 (+ 3021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:39:51.968532 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:39:51.968747 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:51.968781 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 11:39:51.968873 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 11:39:51.968991 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14287,"thread_start_us":13431,"threads_started":1}
I20260808 11:39:51.969512 10951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:51.944875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49858 (local address 127.24.76.66:42815)
0808 11:39:51.957452 (+ 12577us) server_negotiation.cc:206] Beginning negotiation
0808 11:39:51.957460 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:39:51.959019 (+ 1559us) server_negotiation.cc:407] Connection header received
0808 11:39:51.959252 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:39:51.959260 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:39:51.959475 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:39:51.959715 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:39:51.960953 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:51.962665 (+ 1712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:51.966283 (+ 3618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:51.968257 (+ 1974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:51.968404 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:51.968588 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0808 11:39:51.968942 (+ 354us) server_negotiation.cc:299] Negotiation successful
0808 11:39:51.969106 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12405,"thread_start_us":12273,"threads_started":1}
I20260808 11:39:51.970835 10834 tablet_service.cc:1467] Processing CreateTablet for tablet 1c92d73206c342b48a5ef5c25c8ce17e (DEFAULT_TABLE table=test-table [id=7b9db67c37f04279b198e392876d1125]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:39:51.971452 10453 tablet_service.cc:1467] Processing CreateTablet for tablet 1c92d73206c342b48a5ef5c25c8ce17e (DEFAULT_TABLE table=test-table [id=7b9db67c37f04279b198e392876d1125]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:39:51.972604 10834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c92d73206c342b48a5ef5c25c8ce17e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:39:51.972843 10453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c92d73206c342b48a5ef5c25c8ce17e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:39:51.973534 10644 tablet_service.cc:1467] Processing CreateTablet for tablet 1c92d73206c342b48a5ef5c25c8ce17e (DEFAULT_TABLE table=test-table [id=7b9db67c37f04279b198e392876d1125]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:39:51.974941 10644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c92d73206c342b48a5ef5c25c8ce17e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:39:51.985476 10954 tablet_bootstrap.cc:492] T 1c92d73206c342b48a5ef5c25c8ce17e P f36c198c12954896936a8d42700771a2: Bootstrap starting.
I20260808 11:39:51.989086 10954 tablet_bootstrap.cc:654] T 1c92d73206c342b48a5ef5c25c8ce17e P f36c198c12954896936a8d42700771a2: Neither blocks nor log segments found. Creating new log.
I20260808 11:39:51.990502 10954 log.cc:824] T 1c92d73206c342b48a5ef5c25c8ce17e P f36c198c12954896936a8d42700771a2: Log is configured to *not* fsync() on all Append() calls
I20260808 11:39:51.990850 10956 tablet_bootstrap.cc:492] T 1c92d73206c342b48a5ef5c25c8ce17e P 0864cd8a9da0422a9dc422a2dc9c5666: Bootstrap starting.
I20260808 11:39:51.992153 10957 tablet_bootstrap.cc:492] T 1c92d73206c342b48a5ef5c25c8ce17e P 3ef52967ad864360a296dfc9effc0b05: Bootstrap starting.
I20260808 11:39:51.995447 10957 tablet_bootstrap.cc:654] T 1c92d73206c342b48a5ef5c25c8ce17e P 3ef52967ad864360a296dfc9effc0b05: Neither blocks nor log segments found. Creating new log.
I20260808 11:39:51.995528 10956 tablet_bootstrap.cc:654] T 1c92d73206c342b48a5ef5c25c8ce17e P 0864cd8a9da0422a9dc422a2dc9c5666: Neither blocks nor log segments found. Creating new log.
I20260808 11:39:51.995973 10954 tablet_bootstrap.cc:492] T 1c92d73206c342b48a5ef5c25c8ce17e P f36c198c12954896936a8d42700771a2: No bootstrap required, opened a new log
I20260808 11:39:51.996297 10954 ts_tablet_manager.cc:1397] T 1c92d73206c342b48a5ef5c25c8ce17e P f36c198c12954896936a8d42700771a2: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.010s
I20260808 11:39:51.996592 10957 log.cc:824] T 1c92d73206c342b48a5ef5c25c8ce17e P 3ef52967ad864360a296dfc9effc0b05: Log is configured to *not* fsync() on all Append() calls
I20260808 11:39:51.996744 10956 log.cc:824] T 1c92d73206c342b48a5ef5c25c8ce17e P 0864cd8a9da0422a9dc422a2dc9c5666: Log is configured to *not* fsync() on all Append() calls
I20260808 11:39:52.000460 10956 tablet_bootstrap.cc:492] T 1c92d73206c342b48a5ef5c25c8ce17e P 0864cd8a9da0422a9dc422a2dc9c5666: No bootstrap required, opened a new log
I20260808 11:39:52.000737 10956 ts_tablet_manager.cc:1397] T 1c92d73206c342b48a5ef5c25c8ce17e P 0864cd8a9da0422a9dc422a2dc9c5666: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260808 11:39:52.000990 10957 tablet_bootstrap.cc:492] T 1c92d73206c342b48a5ef5c25c8ce17e P 3ef52967ad864360a296dfc9effc0b05: No bootstrap required, opened a new log
I20260808 11:39:52.001256 10957 ts_tablet_manager.cc:1397] T 1c92d73206c342b48a5ef5c25c8ce17e P 3ef52967ad864360a296dfc9effc0b05: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260808 11:39:52.007206 10954 raft_consensus.cc:348] T 1c92d73206c342b48a5ef5c25c8ce17e P f36c198c12954896936a8d42700771a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0864cd8a9da0422a9dc422a2dc9c5666" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 45943 } } peers { permanent_uuid: "f36c198c12954896936a8d42700771a2" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43385 } } peers { permanent_uuid: "3ef52967ad864360a296dfc9effc0b05" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42815 } }
I20260808 11:39:52.007809 10954 raft_consensus.cc:374] T 1c92d73206c342b48a5ef5c25c8ce17e P f36c198c12954896936a8d42700771a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:39:52.007969 10954 raft_consensus.cc:729] T 1c92d73206c342b48a5ef5c25c8ce17e P f36c198c12954896936a8d42700771a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f36c198c12954896936a8d42700771a2, State: Initialized, Role: FOLLOWER
I20260808 11:39:52.008776 10954 consensus_queue.cc:260] T 1c92d73206c342b48a5ef5c25c8ce17e P f36c198c12954896936a8d42700771a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0864cd8a9da0422a9dc422a2dc9c5666" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 45943 } } peers { permanent_uuid: "f36c198c12954896936a8d42700771a2" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43385 } } peers { permanent_uuid: "3ef52967ad864360a296dfc9effc0b05" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42815 } }
I20260808 11:39:52.009406 10956 raft_consensus.cc:348] T 1c92d73206c342b48a5ef5c25c8ce17e P 0864cd8a9da0422a9dc422a2dc9c5666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0864cd8a9da0422a9dc422a2dc9c5666" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 45943 } } peers { permanent_uuid: "f36c198c12954896936a8d42700771a2" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43385 } } peers { permanent_uuid: "3ef52967ad864360a296dfc9effc0b05" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42815 } }
I20260808 11:39:52.009831 10956 raft_consensus.cc:374] T 1c92d73206c342b48a5ef5c25c8ce17e P 0864cd8a9da0422a9dc422a2dc9c5666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:39:52.009963 10956 raft_consensus.cc:729] T 1c92d73206c342b48a5ef5c25c8ce17e P 0864cd8a9da0422a9dc422a2dc9c5666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0864cd8a9da0422a9dc422a2dc9c5666, State: Initialized, Role: FOLLOWER
I20260808 11:39:52.010588 10956 consensus_queue.cc:260] T 1c92d73206c342b48a5ef5c25c8ce17e P 0864cd8a9da0422a9dc422a2dc9c5666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0864cd8a9da0422a9dc422a2dc9c5666" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 45943 } } peers { permanent_uuid: "f36c198c12954896936a8d42700771a2" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43385 } } peers { permanent_uuid: "3ef52967ad864360a296dfc9effc0b05" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42815 } }
I20260808 11:39:52.010816 10954 ts_tablet_manager.cc:1428] T 1c92d73206c342b48a5ef5c25c8ce17e P f36c198c12954896936a8d42700771a2: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260808 11:39:52.012028 10931 heartbeater.cc:503] Master 127.24.76.126:40965 was elected leader, sending a full tablet report...
I20260808 11:39:52.012614 10956 ts_tablet_manager.cc:1428] T 1c92d73206c342b48a5ef5c25c8ce17e P 0864cd8a9da0422a9dc422a2dc9c5666: Time spent starting tablet: real 0.012s user 0.007s sys 0.004s
I20260808 11:39:52.012626 10957 raft_consensus.cc:348] T 1c92d73206c342b48a5ef5c25c8ce17e P 3ef52967ad864360a296dfc9effc0b05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0864cd8a9da0422a9dc422a2dc9c5666" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 45943 } } peers { permanent_uuid: "f36c198c12954896936a8d42700771a2" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43385 } } peers { permanent_uuid: "3ef52967ad864360a296dfc9effc0b05" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42815 } }
I20260808 11:39:52.013075 10957 raft_consensus.cc:374] T 1c92d73206c342b48a5ef5c25c8ce17e P 3ef52967ad864360a296dfc9effc0b05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:39:52.013168 10957 raft_consensus.cc:729] T 1c92d73206c342b48a5ef5c25c8ce17e P 3ef52967ad864360a296dfc9effc0b05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef52967ad864360a296dfc9effc0b05, State: Initialized, Role: FOLLOWER
I20260808 11:39:52.013823 10957 consensus_queue.cc:260] T 1c92d73206c342b48a5ef5c25c8ce17e P 3ef52967ad864360a296dfc9effc0b05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0864cd8a9da0422a9dc422a2dc9c5666" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 45943 } } peers { permanent_uuid: "f36c198c12954896936a8d42700771a2" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43385 } } peers { permanent_uuid: "3ef52967ad864360a296dfc9effc0b05" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42815 } }
I20260808 11:39:52.015261 10735 heartbeater.cc:503] Master 127.24.76.126:40965 was elected leader, sending a full tablet report...
I20260808 11:39:52.016846 10957 ts_tablet_manager.cc:1428] T 1c92d73206c342b48a5ef5c25c8ce17e P 3ef52967ad864360a296dfc9effc0b05: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
W20260808 11:39:52.114651 10546 tablet.cc:2369] T 1c92d73206c342b48a5ef5c25c8ce17e P f36c198c12954896936a8d42700771a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:39:52.117584 10932 tablet.cc:2369] T 1c92d73206c342b48a5ef5c25c8ce17e P 0864cd8a9da0422a9dc422a2dc9c5666: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:39:52.122967 10965 raft_consensus.cc:482] T 1c92d73206c342b48a5ef5c25c8ce17e P 0864cd8a9da0422a9dc422a2dc9c5666 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:39:52.123293 10965 raft_consensus.cc:504] T 1c92d73206c342b48a5ef5c25c8ce17e P 0864cd8a9da0422a9dc422a2dc9c5666 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0864cd8a9da0422a9dc422a2dc9c5666" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 45943 } } peers { permanent_uuid: "f36c198c12954896936a8d42700771a2" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43385 } } peers { permanent_uuid: "3ef52967ad864360a296dfc9effc0b05" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42815 } }
I20260808 11:39:52.125046 10965 leader_election.cc:288] T 1c92d73206c342b48a5ef5c25c8ce17e P 0864cd8a9da0422a9dc422a2dc9c5666 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f36c198c12954896936a8d42700771a2 (127.24.76.65:43385), 3ef52967ad864360a296dfc9effc0b05 (127.24.76.66:42815)
I20260808 11:39:52.131780 10935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:52.125608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:43385 (local address 127.24.76.67:53551)
0808 11:39:52.125758 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 11:39:52.125777 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:39:52.125928 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:39:52.126345 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:39:52.126351 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:39:52.126393 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:39:52.126630 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:39:52.126642 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:39:52.127906 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:39:52.127909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:39:52.129807 (+ 1898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:39:52.129824 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:39:52.131120 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:39:52.131125 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:39:52.131361 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:52.131387 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 11:39:52.131493 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 11:39:52.131588 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":1}
I20260808 11:39:52.131934 10949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:52.125593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:53551 (local address 127.24.76.65:43385)
0808 11:39:52.125796 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 11:39:52.125802 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:39:52.125829 (+ 27us) server_negotiation.cc:407] Connection header received
0808 11:39:52.126099 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:39:52.126104 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:39:52.126187 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:39:52.126284 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:39:52.126776 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:52.127772 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:52.129989 (+ 2217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:52.130947 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:52.131128 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:52.131215 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 11:39:52.131545 (+ 330us) server_negotiation.cc:299] Negotiation successful
0808 11:39:52.131674 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 11:39:52.132951 10951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:52.125549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:43883 (local address 127.24.76.66:42815)
0808 11:39:52.125803 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0808 11:39:52.125808 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:39:52.126842 (+ 1034us) server_negotiation.cc:407] Connection header received
0808 11:39:52.126961 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:39:52.126967 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:39:52.127045 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:39:52.127139 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:39:52.127778 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:52.128596 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:52.130819 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:52.132120 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:52.132272 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:52.132346 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 11:39:52.132594 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 11:39:52.132714 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 11:39:52.133061 10967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:52.125620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:42815 (local address 127.24.76.67:43883)
0808 11:39:52.126746 (+ 1126us) negotiation.cc:107] Waiting for socket to connect
0808 11:39:52.126763 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:39:52.126855 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:39:52.127141 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:39:52.127144 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:39:52.127160 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:39:52.127600 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:39:52.127610 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:39:52.128770 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:39:52.128776 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:39:52.130672 (+ 1896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:39:52.130684 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:39:52.132320 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:39:52.132324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:39:52.132440 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:52.132466 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 11:39:52.132547 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 11:39:52.132661 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":928,"mutex_wait_us":216,"thread_start_us":459,"threads_started":1}
I20260808 11:39:52.132956 10486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c92d73206c342b48a5ef5c25c8ce17e" candidate_uuid: "0864cd8a9da0422a9dc422a2dc9c5666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f36c198c12954896936a8d42700771a2" is_pre_election: true
I20260808 11:39:52.133563 10486 raft_consensus.cc:2454] T 1c92d73206c342b48a5ef5c25c8ce17e P f36c198c12954896936a8d42700771a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0864cd8a9da0422a9dc422a2dc9c5666 in term 0.
I20260808 11:39:52.134410 10765 leader_election.cc:302] T 1c92d73206c342b48a5ef5c25c8ce17e P 0864cd8a9da0422a9dc422a2dc9c5666 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0864cd8a9da0422a9dc422a2dc9c5666, f36c198c12954896936a8d42700771a2; no voters:
I20260808 11:39:52.134297 10673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c92d73206c342b48a5ef5c25c8ce17e" candidate_uuid: "0864cd8a9da0422a9dc422a2dc9c5666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef52967ad864360a296dfc9effc0b05" is_pre_election: true
I20260808 11:39:52.134785 10965 raft_consensus.cc:2781] T 1c92d73206c342b48a5ef5c25c8ce17e P 0864cd8a9da0422a9dc422a2dc9c5666 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:39:52.135000 10965 raft_consensus.cc:482] T 1c92d73206c342b48a5ef5c25c8ce17e P 0864cd8a9da0422a9dc422a2dc9c5666 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:39:52.135128 10965 raft_consensus.cc:3037] T 1c92d73206c342b48a5ef5c25c8ce17e P 0864cd8a9da0422a9dc422a2dc9c5666 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:39:52.134907 10673 raft_consensus.cc:2454] T 1c92d73206c342b48a5ef5c25c8ce17e P 3ef52967ad864360a296dfc9effc0b05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0864cd8a9da0422a9dc422a2dc9c5666 in term 0.
I20260808 11:39:52.137733 10965 raft_consensus.cc:504] T 1c92d73206c342b48a5ef5c25c8ce17e P 0864cd8a9da0422a9dc422a2dc9c5666 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0864cd8a9da0422a9dc422a2dc9c5666" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 45943 } } peers { permanent_uuid: "f36c198c12954896936a8d42700771a2" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43385 } } peers { permanent_uuid: "3ef52967ad864360a296dfc9effc0b05" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42815 } }
I20260808 11:39:52.138442 10965 leader_election.cc:288] T 1c92d73206c342b48a5ef5c25c8ce17e P 0864cd8a9da0422a9dc422a2dc9c5666 [CANDIDATE]: Term 1 election: Requested vote from peers f36c198c12954896936a8d42700771a2 (127.24.76.65:43385), 3ef52967ad864360a296dfc9effc0b05 (127.24.76.66:42815)
I20260808 11:39:52.138720 10486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c92d73206c342b48a5ef5c25c8ce17e" candidate_uuid: "0864cd8a9da0422a9dc422a2dc9c5666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f36c198c12954896936a8d42700771a2"
I20260808 11:39:52.138932 10486 raft_consensus.cc:3037] T 1c92d73206c342b48a5ef5c25c8ce17e P f36c198c12954896936a8d42700771a2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:39:52.138978 10673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c92d73206c342b48a5ef5c25c8ce17e" candidate_uuid: "0864cd8a9da0422a9dc422a2dc9c5666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ef52967ad864360a296dfc9effc0b05"
I20260808 11:39:52.139232 10673 raft_consensus.cc:3037] T 1c92d73206c342b48a5ef5c25c8ce17e P 3ef52967ad864360a296dfc9effc0b05 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:39:52.142475 10486 raft_consensus.cc:2454] T 1c92d73206c342b48a5ef5c25c8ce17e P f36c198c12954896936a8d42700771a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0864cd8a9da0422a9dc422a2dc9c5666 in term 1.
I20260808 11:39:52.142810 10673 raft_consensus.cc:2454] T 1c92d73206c342b48a5ef5c25c8ce17e P 3ef52967ad864360a296dfc9effc0b05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0864cd8a9da0422a9dc422a2dc9c5666 in term 1.
I20260808 11:39:52.142853 10765 leader_election.cc:302] T 1c92d73206c342b48a5ef5c25c8ce17e P 0864cd8a9da0422a9dc422a2dc9c5666 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0864cd8a9da0422a9dc422a2dc9c5666, f36c198c12954896936a8d42700771a2; no voters:
I20260808 11:39:52.143097 10965 raft_consensus.cc:2781] T 1c92d73206c342b48a5ef5c25c8ce17e P 0864cd8a9da0422a9dc422a2dc9c5666 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:39:52.143615 10965 raft_consensus.cc:686] T 1c92d73206c342b48a5ef5c25c8ce17e P 0864cd8a9da0422a9dc422a2dc9c5666 [term 1 LEADER]: Becoming Leader. State: Replica: 0864cd8a9da0422a9dc422a2dc9c5666, State: Running, Role: LEADER
I20260808 11:39:52.144109 10965 consensus_queue.cc:237] T 1c92d73206c342b48a5ef5c25c8ce17e P 0864cd8a9da0422a9dc422a2dc9c5666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0864cd8a9da0422a9dc422a2dc9c5666" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 45943 } } peers { permanent_uuid: "f36c198c12954896936a8d42700771a2" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43385 } } peers { permanent_uuid: "3ef52967ad864360a296dfc9effc0b05" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42815 } }
I20260808 11:39:52.149326 10224 catalog_manager.cc:5495] T 1c92d73206c342b48a5ef5c25c8ce17e P 0864cd8a9da0422a9dc422a2dc9c5666 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0864cd8a9da0422a9dc422a2dc9c5666 (127.24.76.67). New cstate: current_term: 1 leader_uuid: "0864cd8a9da0422a9dc422a2dc9c5666" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0864cd8a9da0422a9dc422a2dc9c5666" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 45943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f36c198c12954896936a8d42700771a2" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef52967ad864360a296dfc9effc0b05" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42815 } health_report { overall_health: UNKNOWN } } }
W20260808 11:39:52.188671 10736 tablet.cc:2369] T 1c92d73206c342b48a5ef5c25c8ce17e P 3ef52967ad864360a296dfc9effc0b05: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:39:52.205646 10948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:52.199814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39374 (local address 127.24.76.67:45943)
0808 11:39:52.200021 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 11:39:52.200027 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:39:52.200094 (+ 67us) server_negotiation.cc:407] Connection header received
0808 11:39:52.200318 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:39:52.200322 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:39:52.200404 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:39:52.200521 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:39:52.200851 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:52.201808 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:52.203577 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:52.204044 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:52.204142 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:52.204710 (+ 568us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:39:52.205226 (+ 516us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:39:52.205330 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 11:39:52.205387 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 11:39:52.205453 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 11:39:52.214439 10673 raft_consensus.cc:1261] T 1c92d73206c342b48a5ef5c25c8ce17e P 3ef52967ad864360a296dfc9effc0b05 [term 1 FOLLOWER]: Refusing update from remote peer 0864cd8a9da0422a9dc422a2dc9c5666: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:39:52.214504 10486 raft_consensus.cc:1261] T 1c92d73206c342b48a5ef5c25c8ce17e P f36c198c12954896936a8d42700771a2 [term 1 FOLLOWER]: Refusing update from remote peer 0864cd8a9da0422a9dc422a2dc9c5666: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:39:52.215495 10969 consensus_queue.cc:1035] T 1c92d73206c342b48a5ef5c25c8ce17e P 0864cd8a9da0422a9dc422a2dc9c5666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f36c198c12954896936a8d42700771a2" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:39:52.215960 10965 consensus_queue.cc:1035] T 1c92d73206c342b48a5ef5c25c8ce17e P 0864cd8a9da0422a9dc422a2dc9c5666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ef52967ad864360a296dfc9effc0b05" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 42815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:39:52.241946 10223 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:39:52.242244 10223 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:39:52.245941 10223 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bf8d8c21477f401684f328da22959a54: Sending DeleteTablet for 3 replicas of tablet 1c92d73206c342b48a5ef5c25c8ce17e
I20260808 11:39:52.247052 10834 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c92d73206c342b48a5ef5c25c8ce17e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:39:52 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:39372
I20260808 11:39:52.247282 10453 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c92d73206c342b48a5ef5c25c8ce17e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:39:52 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:46156
I20260808 11:39:52.247316 10644 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c92d73206c342b48a5ef5c25c8ce17e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:39:52 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:49858
I20260808 11:39:52.250070 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 10348
I20260808 11:39:52.258419 10989 tablet_replica.cc:330] T 1c92d73206c342b48a5ef5c25c8ce17e P 3ef52967ad864360a296dfc9effc0b05: stopping tablet replica
I20260808 11:39:52.258955 10989 raft_consensus.cc:2229] T 1c92d73206c342b48a5ef5c25c8ce17e P 3ef52967ad864360a296dfc9effc0b05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:39:52.259598 10989 raft_consensus.cc:2258] T 1c92d73206c342b48a5ef5c25c8ce17e P 3ef52967ad864360a296dfc9effc0b05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:39:52.260430 10986 tablet_replica.cc:330] T 1c92d73206c342b48a5ef5c25c8ce17e P 0864cd8a9da0422a9dc422a2dc9c5666: stopping tablet replica
I20260808 11:39:52.260810 10986 raft_consensus.cc:2229] T 1c92d73206c342b48a5ef5c25c8ce17e P 0864cd8a9da0422a9dc422a2dc9c5666 [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:39:52.261459 10986 raft_consensus.cc:2258] T 1c92d73206c342b48a5ef5c25c8ce17e P 0864cd8a9da0422a9dc422a2dc9c5666 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:39:52.263480 10989 ts_tablet_manager.cc:1905] T 1c92d73206c342b48a5ef5c25c8ce17e P 3ef52967ad864360a296dfc9effc0b05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:39:52.263700 10986 ts_tablet_manager.cc:1905] T 1c92d73206c342b48a5ef5c25c8ce17e P 0864cd8a9da0422a9dc422a2dc9c5666: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:39:52.272315 10986 ts_tablet_manager.cc:1918] T 1c92d73206c342b48a5ef5c25c8ce17e P 0864cd8a9da0422a9dc422a2dc9c5666: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:39:52.272341 10989 ts_tablet_manager.cc:1918] T 1c92d73206c342b48a5ef5c25c8ce17e P 3ef52967ad864360a296dfc9effc0b05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:39:52.272502 10989 log.cc:1196] T 1c92d73206c342b48a5ef5c25c8ce17e P 3ef52967ad864360a296dfc9effc0b05: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-1/wal/wals/1c92d73206c342b48a5ef5c25c8ce17e
I20260808 11:39:52.272734 10986 log.cc:1196] T 1c92d73206c342b48a5ef5c25c8ce17e P 0864cd8a9da0422a9dc422a2dc9c5666: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-2/wal/wals/1c92d73206c342b48a5ef5c25c8ce17e
I20260808 11:39:52.273110 10989 ts_tablet_manager.cc:1939] T 1c92d73206c342b48a5ef5c25c8ce17e P 3ef52967ad864360a296dfc9effc0b05: Deleting consensus metadata
I20260808 11:39:52.273763 10986 ts_tablet_manager.cc:1939] T 1c92d73206c342b48a5ef5c25c8ce17e P 0864cd8a9da0422a9dc422a2dc9c5666: Deleting consensus metadata
I20260808 11:39:52.275095 10204 catalog_manager.cc:4841] TS 3ef52967ad864360a296dfc9effc0b05 (127.24.76.66:42815): tablet 1c92d73206c342b48a5ef5c25c8ce17e (table test-table [id=7b9db67c37f04279b198e392876d1125]) successfully deleted
I20260808 11:39:52.275848 10205 catalog_manager.cc:4841] TS 0864cd8a9da0422a9dc422a2dc9c5666 (127.24.76.67:45943): tablet 1c92d73206c342b48a5ef5c25c8ce17e (table test-table [id=7b9db67c37f04279b198e392876d1125]) successfully deleted
I20260808 11:39:52.275919 10987 tablet_replica.cc:330] T 1c92d73206c342b48a5ef5c25c8ce17e P f36c198c12954896936a8d42700771a2: stopping tablet replica
I20260808 11:39:52.276391 10987 raft_consensus.cc:2229] T 1c92d73206c342b48a5ef5c25c8ce17e P f36c198c12954896936a8d42700771a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:39:52.277064 10987 raft_consensus.cc:2258] T 1c92d73206c342b48a5ef5c25c8ce17e P f36c198c12954896936a8d42700771a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:39:52.278813 10987 ts_tablet_manager.cc:1905] T 1c92d73206c342b48a5ef5c25c8ce17e P f36c198c12954896936a8d42700771a2: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 08 11:39:52 dist-test-slave-tc2s krb5kdc[9512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.65@KRBTEST.COM
I20260808 11:39:52.284417 10987 ts_tablet_manager.cc:1918] T 1c92d73206c342b48a5ef5c25c8ce17e P f36c198c12954896936a8d42700771a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:39:52.284574 10987 log.cc:1196] T 1c92d73206c342b48a5ef5c25c8ce17e P f36c198c12954896936a8d42700771a2: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786189042529715-24881-0/minicluster-data/ts-0/wal/wals/1c92d73206c342b48a5ef5c25c8ce17e
I20260808 11:39:52.285066 10987 ts_tablet_manager.cc:1939] T 1c92d73206c342b48a5ef5c25c8ce17e P f36c198c12954896936a8d42700771a2: Deleting consensus metadata
I20260808 11:39:52.286608 10206 catalog_manager.cc:4841] TS f36c198c12954896936a8d42700771a2 (127.24.76.65:43385): tablet 1c92d73206c342b48a5ef5c25c8ce17e (table test-table [id=7b9db67c37f04279b198e392876d1125]) successfully deleted
I20260808 11:39:52.290635 10949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:52.251588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46172 (local address 127.24.76.65:43385)
0808 11:39:52.251761 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 11:39:52.251768 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:39:52.270654 (+ 18886us) server_negotiation.cc:407] Connection header received
0808 11:39:52.270861 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:39:52.270867 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:39:52.270963 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:39:52.271082 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:39:52.272962 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:52.274144 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:52.276015 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:52.276598 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:52.276690 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:52.282760 (+ 6070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:39:52.282775 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:39:52.282800 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:39:52.282843 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:39:52.287625 (+ 4782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:39:52.288271 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:39:52.288279 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:39:52.288283 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:39:52.288521 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:39:52.288903 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:39:52.288909 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:39:52.288912 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:39:52.289376 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:39:52.289744 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0808 11:39:52.289978 (+ 234us) server_negotiation.cc:299] Negotiation successful
0808 11:39:52.290231 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 11:39:52.291340 10533 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:39:52.397076 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 10348
I20260808 11:39:52.421362 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 10554
Aug 08 11:39:52 dist-test-slave-tc2s krb5kdc[9512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.66@KRBTEST.COM
I20260808 11:39:52.436813 10951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:52.422150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49860 (local address 127.24.76.66:42815)
0808 11:39:52.422695 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0808 11:39:52.422699 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:39:52.422714 (+ 15us) server_negotiation.cc:407] Connection header received
0808 11:39:52.422767 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:39:52.422771 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:39:52.422842 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:39:52.422928 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:39:52.424252 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:52.425004 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:52.426310 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:52.426708 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:52.426836 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:52.431281 (+ 4445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:39:52.431296 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:39:52.431312 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:39:52.431356 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:39:52.434145 (+ 2789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:39:52.434794 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:39:52.434802 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:39:52.434804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:39:52.434939 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:39:52.435293 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:39:52.435297 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:39:52.435299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:39:52.435681 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:39:52.435956 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0808 11:39:52.436272 (+ 316us) server_negotiation.cc:299] Negotiation successful
0808 11:39:52.436479 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432}
I20260808 11:39:52.437453 10728 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:39:52.528813 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 10554
I20260808 11:39:52.563037 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 10746
Aug 08 11:39:52 dist-test-slave-tc2s krb5kdc[9512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.67@KRBTEST.COM
I20260808 11:39:52.579620 10948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:39:52.563871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39388 (local address 127.24.76.67:45943)
0808 11:39:52.564093 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 11:39:52.564098 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:39:52.564121 (+ 23us) server_negotiation.cc:407] Connection header received
0808 11:39:52.564254 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:39:52.564259 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:39:52.564333 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:39:52.564535 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:39:52.565878 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:52.566733 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:52.568058 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:39:52.568510 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:39:52.568644 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:39:52.573249 (+ 4605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:39:52.573265 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:39:52.573282 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:39:52.573329 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:39:52.576532 (+ 3203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:39:52.577428 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:39:52.577435 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:39:52.577438 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:39:52.577592 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:39:52.578040 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:39:52.578043 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:39:52.578045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:39:52.578414 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:39:52.578661 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0808 11:39:52.579064 (+ 403us) server_negotiation.cc:299] Negotiation successful
0808 11:39:52.579225 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260808 11:39:52.580488 10921 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:39:52.703869 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 10746
I20260808 11:39:52.730381 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 10176
I20260808 11:39:52.731530 10273 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:39:52.849010 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 10176
I20260808 11:39:52.878386 24881 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 11:39:53.222421 24881 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 11:39:53.222682 24881 mini_ranger.cc:66] Stopping Ranger...
I20260808 11:39:53.576858 24881 mini_ranger.cc:68] Stopped Ranger
2026-08-08 11:39:53.577 UTC [9557] LOG: received smart shutdown request
2026-08-08 11:39:53.580 UTC [9557] LOG: background worker "logical replication launcher" (PID 9576) exited with exit code 1
2026-08-08 11:39:53.581 UTC [9567] LOG: shutting down
2026-08-08 11:39:53.713 UTC [9557] LOG: database system is shut down
2026-08-08T11:39:53Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (51348 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:39:53 dist-test-slave-tc2s krb5kdc[11011](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:39:53 dist-test-slave-tc2s krb5kdc[11011](info): set up 2 sockets
Aug 08 11:39:53 dist-test-slave-tc2s krb5kdc[11011](info): commencing operation
krb5kdc: starting...
W20260808 11:39:55.042598 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.180s user 0.000s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:39:55 dist-test-slave-tc2s krb5kdc[11011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:39:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:39:55Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.24.76.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.24.76.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.24.76.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/rangeradmin_127.24.76.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.24.76.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/rangeradmin_127.24.76.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.24.76.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.24.76.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.24.76.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/rangerlookup_127.24.76.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.24.76.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/rangerlookup_127.24.76.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.76.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.24.76.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.76.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/HTTP_127.24.76.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.76.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/HTTP_127.24.76.84@KRBTEST.COM.keytab.
I20260808 11:39:55.385870 24881 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskryQpdG/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/postgres -l logfile start
2026-08-08 11:39:57.626 UTC [11051] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 11:39:57.627 UTC [11051] LOG: listening on IPv4 address "127.24.76.84", port 41791
2026-08-08 11:39:57.630 UTC [11051] LOG: listening on Unix socket "/tmp/.s.PGSQL.41791"
2026-08-08 11:39:57.651 UTC [11054] LOG: database system was shut down at 2026-08-08 11:39:56 UTC
2026-08-08 11:39:57.655 UTC [11051] LOG: database system is ready to accept connections
I20260808 11:39:58.731194 24881 mini_postgres.cc:96] Postgres bound to 41791
2026-08-08 11:39:58.758 UTC [11069] FATAL: database "slave" does not exist
127.24.76.84:41791 - accepting connections
I20260808 11:39:58.759155 24881 mini_ranger.cc:161] Starting Ranger...
I20260808 11:39:58.786602 24881 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 11:39:59.205446 24881 mini_ranger.cc:87] Created ranger Postgres database
I20260808 11:39:59.205562 24881 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ranger-admin
2026-08-08 11:39:59,490 [I] DB FLAVOR :POSTGRES
2026-08-08 11:39:59,490 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 11:39:59,491 [I] Checking connection..
2026-08-08 11:39:59,491 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:41791/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 11:39:59,745 [I] Checking connection passed.
2026-08-08 11:39:59,745 [I] --------- Verifying version history table ---------
2026-08-08 11:39:59,745 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:41791/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 11:40:00,001 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 11:40:00,002 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:41791/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 11:40:00,257 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 11:40:00,258 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 11:40:00,258 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:41791/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 11:40:00.497 UTC [11169] WARNING: there is no transaction in progress
2026-08-08 11:40:00,504 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 11:40:00,504 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:41791/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 11:40:00,755 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 11:40:00,755 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 11:40:00,755 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:41791/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 11:40:01,008 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:41791/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 11:40:01,229 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:41791/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-tc2s.c.gcp-upstream.internal','N') ;"
2026-08-08 11:40:01,473 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:41791/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 11:40:01,731 [I] Table x_portal_user does not exist in database ranger
2026-08-08 11:40:01,732 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:41791/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 11:40:01,989 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 11:40:01,990 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:41791/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 11:40:02,240 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:41791/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 11:40:02,493 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 11:40:02,493 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:41791/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 11:40:03.362 UTC [11337] WARNING: there is no transaction in progress
2026-08-08 11:40:03.376 UTC [11337] WARNING: there is no transaction in progress
2026-08-08 11:40:03.390 UTC [11337] WARNING: there is no transaction in progress
2026-08-08 11:40:03.403 UTC [11337] WARNING: there is no transaction in progress
2026-08-08 11:40:03.419 UTC [11337] WARNING: there is no transaction in progress
2026-08-08 11:40:03.437 UTC [11337] WARNING: there is no transaction in progress
2026-08-08 11:40:03.496 UTC [11337] WARNING: there is no transaction in progress
2026-08-08 11:40:03.503 UTC [11337] WARNING: there is no transaction in progress
2026-08-08 11:40:03.515 UTC [11337] WARNING: there is no transaction in progress
2026-08-08 11:40:03.524 UTC [11337] WARNING: there is no transaction in progress
2026-08-08 11:40:03.533 UTC [11337] WARNING: there is no transaction in progress
2026-08-08 11:40:03.545 UTC [11337] WARNING: there is no transaction in progress
2026-08-08 11:40:03.554 UTC [11337] WARNING: there is no transaction in progress
2026-08-08 11:40:03.559 UTC [11337] WARNING: there is no transaction in progress
2026-08-08 11:40:03.565 UTC [11337] WARNING: there is no transaction in progress
2026-08-08 11:40:03.570 UTC [11337] WARNING: there is no transaction in progress
2026-08-08 11:40:04,127 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 11:40:04,127 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:41791/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 11:40:04,377 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 11:40:04,377 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:41791/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 11:40:04,623 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 11:40:04,623 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:41791/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 11:40:04,880 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 11:40:04,880 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:41791/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 11:40:05,125 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:41791/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 11:40:05,378 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:41791/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-tc2s.c.gcp-upstream.internal';"
2026-08-08 11:40:05,633 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 11:40:05,634 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:41791/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 11:40:05,886 [I] DB_PATCHES have already been applied
I20260808 11:40:05.892426 24881 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ranger-admin:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskryQpdG/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 11:40:05.892517 24881 mini_ranger.cc:193] Using host: 127.24.76.84
I20260808 11:40:05.912428 24881 mini_ranger.cc:227] Ranger admin URL: http://127.24.76.84:55869
Aug 08, 2026 11:40:06 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 11:40:06 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 11:40:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 11:40:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 11:40:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 11:40:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 11:40:06 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 11:40:07.163187 24881 mini_ranger.cc:160] Time spent starting Ranger: real 8.404s user 0.003s sys 0.054s
Aug 08, 2026 11:40:25 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 11:40:25 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 08, 2026 11:40:25 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 11:40:25 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 11:40:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 11:40:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 11:40:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 11:40:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 11:40:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 11:40:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 11:40:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 11:40:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 11:40:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 11:40:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 11:40:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 11:40:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 11:40:25 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 11:40:26 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 11:40:27.291365 24881 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.24.76.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.24.76.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.24.76.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/rangerkms_127.24.76.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.24.76.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/rangerkms_127.24.76.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.76.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.24.76.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.76.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/HTTP_127.24.76.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.76.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/HTTP_127.24.76.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 11:40:27 dist-test-slave-tc2s krb5kdc[11011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189227, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 11:40:27.471717 24881 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 11:40:27.493884 24881 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 11:40:28.225214 24881 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 11:40:28.225329 24881 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ranger-kms
I20260808 11:40:28.225392 24881 mini_ranger_kms.cc:227] Using postgres at 127.24.76.84:41791
2026-08-08 11:40:28,588 [I] DB FLAVOR :POSTGRES
2026-08-08 11:40:28,589 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 11:40:28,589 [I] Checking connection
2026-08-08 11:40:28,589 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:41791/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 11:40:28,831 [I] connection success
2026-08-08 11:40:28,831 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 11:40:28,832 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 11:40:28,832 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:41791/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 11:40:29,057 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 11:40:29,058 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 11:40:29,058 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 11:40:29,058 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.76.84:41791/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 11:40:29,312 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 11:40:29.750063 24881 mini_ranger_kms.cc:310] Created kms service
I20260808 11:40:29.947655 24881 mini_ranger_kms.cc:326] Created kudu user
I20260808 11:40:29.988706 24881 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 11:40:29 dist-test-slave-tc2s krb5kdc[11011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189227, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.24.76.84@KRBTEST.COM
I20260808 11:40:30.395550 24881 mini_ranger_kms.cc:384] Added ranger policy
I20260808 11:40:30.395891 24881 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ranger-kms:/tmp/dist-test-taskryQpdG/build/asan/bin/postgresql.jar:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskryQpdG/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 11:40:30.395960 24881 mini_ranger_kms.cc:240] Using host: 127.24.76.84
I20260808 11:40:30.416718 24881 mini_ranger_kms.cc:276] Ranger KMS PID: 11602
I20260808 11:40:30.416839 24881 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.24.76.84:40693
Aug 08, 2026 11:40:31 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 11:40:31 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 11:40:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 11:40:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 11:40:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 11:40:31 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 11:40:32 dist-test-slave-tc2s krb5kdc[11011](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786189232, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.76.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 11:40:32 dist-test-slave-tc2s krb5kdc[11011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189232, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.76.84@KRBTEST.COM for HTTP/127.24.76.84@KRBTEST.COM
I20260808 11:40:32.954632 24881 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.483s user 0.080s sys 0.310s
I20260808 11:40:32.954876 24881 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 11:40:32.955045 24881 mini_ranger_kms.cc:401] 127.24.76.84:40693/kms/v1/keys
Aug 08 11:40:33 dist-test-slave-tc2s krb5kdc[11011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189232, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.76.84@KRBTEST.COM for HTTP/127.24.76.84@KRBTEST.COM
Aug 08 11:40:35 dist-test-slave-tc2s krb5kdc[11011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.24.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
I20260808 11:40:35.724596 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:45843
--webserver_interface=127.24.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:39149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:45843
--ranger_config_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.76.84:40693/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:40:36.054675 11686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:40:36.054936 11686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:40:36.055048 11686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:40:36.062587 11686 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:40:36.062681 11686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:40:36.062749 11686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:40:36.062821 11686 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:40:36.062881 11686 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:40:36.072743 11686 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:39149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/master-0/wal
--ranger_kms_url=127.24.76.84:40693/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:45843
--ranger_config_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:45843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:40:36.073995 11686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:40:36.075278 11686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:40:36.087040 11692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:36.087319 11693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:36.087458 11695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:40:36.087780 11686 server_base.cc:1034] running on GCE node
I20260808 11:40:36.088559 11686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:40:36.089972 11686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:40:36.091184 11686 hybrid_clock.cc:648] HybridClock initialized: now 1786189236091145 us; error 34 us; skew 500 ppm
I20260808 11:40:36.091512 11686 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:40:36 dist-test-slave-tc2s krb5kdc[11011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189236, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:40:36.094791 11686 init.cc:377] Logged in from keytab as kudu/127.24.76.126@KRBTEST.COM (short username kudu)
I20260808 11:40:36.096419 11686 webserver.cc:460] Webserver started at http://127.24.76.126:36805/ using document root <none> and password file <none>
I20260808 11:40:36.097200 11686 fs_manager.cc:359] Metadata directory not provided
I20260808 11:40:36.097325 11686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:40:36.097656 11686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 11:40:36 dist-test-slave-tc2s krb5kdc[11011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189236, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for HTTP/127.24.76.84@KRBTEST.COM
I20260808 11:40:36.155190 11686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "5297282530a04795bf73dafd3716fee2"
format_stamp: "Formatted at 2026-08-08 11:40:36 on dist-test-slave-tc2s"
server_key: "f32d5c1e5f91e53d2212f4462b1a0705"
server_key_iv: "b9330846aa967b5f74ccacdd4f209c9e"
server_key_version: "kuduclusterkey@0"
I20260808 11:40:36.156378 11686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "5297282530a04795bf73dafd3716fee2"
format_stamp: "Formatted at 2026-08-08 11:40:36 on dist-test-slave-tc2s"
server_key: "f32d5c1e5f91e53d2212f4462b1a0705"
server_key_iv: "b9330846aa967b5f74ccacdd4f209c9e"
server_key_version: "kuduclusterkey@0"
I20260808 11:40:36.163432 11686 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260808 11:40:36.183383 11708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:36.185014 11686 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:40:36.185207 11686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "5297282530a04795bf73dafd3716fee2"
format_stamp: "Formatted at 2026-08-08 11:40:36 on dist-test-slave-tc2s"
server_key: "f32d5c1e5f91e53d2212f4462b1a0705"
server_key_iv: "b9330846aa967b5f74ccacdd4f209c9e"
server_key_version: "kuduclusterkey@0"
I20260808 11:40:36.185459 11686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:40:36.219003 11686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:40:36.223599 11686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:40:36.223927 11686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:40:36.242129 11780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:45843 every 8 connection(s)
I20260808 11:40:36.242137 11686 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:45843
I20260808 11:40:36.243724 11686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:40:36.247527 11785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:40:36.252482 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 11686
I20260808 11:40:36.252709 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:40:36.253504 24881 external_mini_cluster.cc:1428] Setting key d907763475bbcf170838de6c01302d2f
I20260808 11:40:36.260049 11785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2: Bootstrap starting.
I20260808 11:40:36.264287 11785 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2: Neither blocks nor log segments found. Creating new log.
I20260808 11:40:36.265750 11785 log.cc:824] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2: Log is configured to *not* fsync() on all Append() calls
I20260808 11:40:36.273432 11785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2: No bootstrap required, opened a new log
I20260808 11:40:36.282773 11785 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5297282530a04795bf73dafd3716fee2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 45843 } }
I20260808 11:40:36.283242 11785 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:40:36.283346 11785 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5297282530a04795bf73dafd3716fee2, State: Initialized, Role: FOLLOWER
I20260808 11:40:36.284121 11785 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5297282530a04795bf73dafd3716fee2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 45843 } }
I20260808 11:40:36.284381 11785 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:40:36.284520 11785 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:40:36.284685 11785 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2 [term 0 FOLLOWER]: Advancing to term 1
Aug 08 11:40:36 dist-test-slave-tc2s krb5kdc[11011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:40:36.287369 11785 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5297282530a04795bf73dafd3716fee2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 45843 } }
I20260808 11:40:36.287962 11785 leader_election.cc:302] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5297282530a04795bf73dafd3716fee2; no voters:
I20260808 11:40:36.288645 11785 leader_election.cc:288] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:40:36.288856 11792 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:40:36.292207 11785 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:40:36.296125 11790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:36.255678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41482 (local address 127.24.76.126:45843)
0808 11:40:36.270798 (+ 15120us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:36.270810 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:36.270852 (+ 42us) server_negotiation.cc:407] Connection header received
0808 11:40:36.273124 (+ 2272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:36.273152 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:36.273628 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:36.274073 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:36.275743 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:36.277779 (+ 2036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:36.279630 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:36.280711 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:36.280903 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:36.286094 (+ 5191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:36.286105 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:36.286124 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:36.286171 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:36.290211 (+ 4040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:36.291056 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:36.291064 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:36.291073 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:36.291270 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:36.291817 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:36.291822 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:36.291826 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:36.292545 (+ 719us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:36.292836 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:36.294306 (+ 1470us) server_negotiation.cc:299] Negotiation successful
0808 11:40:36.294742 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14787,"thread_start_us":14006,"threads_started":1}
I20260808 11:40:36.297204 11792 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2 [term 1 LEADER]: Becoming Leader. State: Replica: 5297282530a04795bf73dafd3716fee2, State: Running, Role: LEADER
I20260808 11:40:36.297832 11792 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5297282530a04795bf73dafd3716fee2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 45843 } }
I20260808 11:40:36.309746 11793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5297282530a04795bf73dafd3716fee2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5297282530a04795bf73dafd3716fee2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 45843 } } }
I20260808 11:40:36.310258 11793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2 [sys.catalog]: This master's current role is: LEADER
I20260808 11:40:36.318619 11795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5297282530a04795bf73dafd3716fee2. Latest consensus state: current_term: 1 leader_uuid: "5297282530a04795bf73dafd3716fee2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5297282530a04795bf73dafd3716fee2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 45843 } } }
I20260808 11:40:36.318874 11803 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:40:36.318929 11795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2 [sys.catalog]: This master's current role is: LEADER
I20260808 11:40:36.325690 11803 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:40:36.351153 11803 catalog_manager.cc:1347] Generated new cluster ID: c5d5b0dd58bf4f13a427358dde354270
I20260808 11:40:36.351289 11803 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:40:36.352288 11785 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 11:40:36.383886 11803 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:40:36.386211 11803 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:40:36.406999 11803 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2: Generated new TSK 0
I20260808 11:40:36.408401 11803 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab.
Entry for principal kudu/127.24.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab.
I20260808 11:40:37.250665 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.76.84:40693/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.65:0
--local_ip_for_outbound_sockets=127.24.76.65
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:45843
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:39149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 11:40:37 dist-test-slave-tc2s krb5kdc[11011](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786189237, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 11:40:37.685575 11850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:40:37.685884 11850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:40:37.686045 11850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:40:37.693787 11850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:40:37.693997 11850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:40:37.694206 11850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.65
I20260808 11:40:37.709342 11850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:39149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.24.76.84:40693/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:45843
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.65
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:40:37.711208 11850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:40:37.712949 11850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:40:37.731161 11872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:37.731571 11868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:37.735741 11875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:40:37.735841 11850 server_base.cc:1034] running on GCE node
I20260808 11:40:37.737331 11850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:40:37.738644 11850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:40:37.739923 11850 hybrid_clock.cc:648] HybridClock initialized: now 1786189237739831 us; error 84 us; skew 500 ppm
I20260808 11:40:37.740245 11850 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:40:37 dist-test-slave-tc2s krb5kdc[11011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189237, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:40:37.747385 11850 init.cc:377] Logged in from keytab as kudu/127.24.76.65@KRBTEST.COM (short username kudu)
I20260808 11:40:37.752075 11850 webserver.cc:460] Webserver started at http://127.24.76.65:34381/ using document root <none> and password file <none>
I20260808 11:40:37.753052 11850 fs_manager.cc:359] Metadata directory not provided
I20260808 11:40:37.753232 11850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:40:37.753660 11850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 11:40:37 dist-test-slave-tc2s krb5kdc[11011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189237, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for HTTP/127.24.76.84@KRBTEST.COM
I20260808 11:40:37.773162 11850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-0/data/instance:
uuid: "2f089cd9b5f44ef981c66348b68e211f"
format_stamp: "Formatted at 2026-08-08 11:40:37 on dist-test-slave-tc2s"
server_key: "28f9394820d54ff894f0d833796fcd29"
server_key_iv: "2644780f0ce4be6cdb139e699e01b18f"
server_key_version: "kuduclusterkey@0"
I20260808 11:40:37.774614 11850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance:
uuid: "2f089cd9b5f44ef981c66348b68e211f"
format_stamp: "Formatted at 2026-08-08 11:40:37 on dist-test-slave-tc2s"
server_key: "28f9394820d54ff894f0d833796fcd29"
server_key_iv: "2644780f0ce4be6cdb139e699e01b18f"
server_key_version: "kuduclusterkey@0"
I20260808 11:40:37.788059 11850 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.004s sys 0.008s
I20260808 11:40:37.822597 11887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:37.827282 11850 fs_manager.cc:725] Time spent opening block manager: real 0.021s user 0.005s sys 0.011s
I20260808 11:40:37.827587 11850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-0/wal
uuid: "2f089cd9b5f44ef981c66348b68e211f"
format_stamp: "Formatted at 2026-08-08 11:40:37 on dist-test-slave-tc2s"
server_key: "28f9394820d54ff894f0d833796fcd29"
server_key_iv: "2644780f0ce4be6cdb139e699e01b18f"
server_key_version: "kuduclusterkey@0"
I20260808 11:40:37.827945 11850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 08 11:40:37 dist-test-slave-tc2s krb5kdc[11011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189237, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for HTTP/127.24.76.84@KRBTEST.COM
I20260808 11:40:37.931936 11850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:40:37.939594 11850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:40:37.940026 11850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:40:37.941099 11850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:40:37.942603 11850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:40:37.942687 11850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:37.942838 11850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:40:37.942894 11850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:38.020594 11850 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.65:44891
I20260808 11:40:38.020617 12073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.65:44891 every 8 connection(s)
I20260808 11:40:38.022081 11850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
I20260808 11:40:38.023512 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 11850
I20260808 11:40:38.023732 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance
I20260808 11:40:38.024576 24881 external_mini_cluster.cc:1428] Setting key 02d313620aff65d2bedaf2195345e703
Aug 08 11:40:38 dist-test-slave-tc2s krb5kdc[11011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189237, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:40:38.075408 12079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:38.028391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:35053 (local address 127.24.76.126:45843)
0808 11:40:38.050882 (+ 22491us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:38.050890 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:38.055050 (+ 4160us) server_negotiation.cc:407] Connection header received
0808 11:40:38.057362 (+ 2312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:38.057368 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:38.057471 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:38.057565 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:38.059903 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:38.061111 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:38.063398 (+ 2287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:38.063981 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:38.064071 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:38.068481 (+ 4410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:38.068488 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:38.068505 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:38.068546 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:38.071859 (+ 3313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:38.072727 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:38.072733 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:38.072736 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:38.072878 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:38.073384 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:38.073389 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:38.073392 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:38.073730 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:38.073859 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:38.074805 (+ 946us) server_negotiation.cc:299] Negotiation successful
0808 11:40:38.075030 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22324,"thread_start_us":22529,"threads_started":1}
WARNING: no policy specified for kudu/127.24.76.66@KRBTEST.COM; defaulting to no policy
I20260808 11:40:38.077852 12078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:38.027748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:45843 (local address 127.24.76.65:35053)
0808 11:40:38.054552 (+ 26804us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:38.054608 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:38.057023 (+ 2415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:38.058155 (+ 1132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:38.058172 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:38.058820 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:40:38.059656 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:38.059686 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:38.061331 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:38.061336 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:38.063234 (+ 1898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:38.063249 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:38.064439 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:38.064445 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:38.064736 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:38.065447 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:40:38.065474 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:40:38.068234 (+ 2760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:40:38.072032 (+ 3798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:38.072040 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:38.072057 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:38.072591 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:38.073017 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:38.073022 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:38.073025 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:38.073268 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:38.073969 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:40:38.073977 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:40:38.074375 (+ 398us) client_negotiation.cc:769] Sending connection context
0808 11:40:38.075993 (+ 1618us) client_negotiation.cc:240] Negotiation successful
0808 11:40:38.076535 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26454,"thread_start_us":26226,"threads_started":1}
I20260808 11:40:38.079708 12076 heartbeater.cc:347] Connected to a master server at 127.24.76.126:45843
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.66@KRBTEST.COM" created.
I20260808 11:40:38.080060 12076 heartbeater.cc:464] Registering TS with master...
I20260808 11:40:38.080936 12076 heartbeater.cc:511] Master 127.24.76.126:45843 requested a full tablet report, sending...
I20260808 11:40:38.086967 11733 ts_manager.cc:194] Registered new tserver with Master: 2f089cd9b5f44ef981c66348b68e211f (127.24.76.65:44891)
I20260808 11:40:38.088871 11733 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.65@KRBTEST.COM'} at 127.24.76.65:35053
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab.
Entry for principal kudu/127.24.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab.
I20260808 11:40:38.119678 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.76.84:40693/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.66:0
--local_ip_for_outbound_sockets=127.24.76.66
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:45843
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:39149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 11:40:38 dist-test-slave-tc2s krb5kdc[11011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189237, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for HTTP/127.24.76.84@KRBTEST.COM
W20260808 11:40:38.609395 12082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:40:38.609690 12082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:40:38.609874 12082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:40:38.617141 12082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:40:38.617256 12082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:40:38.617425 12082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.66
I20260808 11:40:38.628036 12082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:39149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.24.76.84:40693/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:45843
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.66
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:40:38.629376 12082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:40:38.630699 12082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:40:38.646663 12107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:38.650717 12106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:38.653713 12114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:40:38.654707 12082 server_base.cc:1034] running on GCE node
I20260808 11:40:38.655375 12082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:40:38.656371 12082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:40:38.657629 12082 hybrid_clock.cc:648] HybridClock initialized: now 1786189238657570 us; error 49 us; skew 500 ppm
I20260808 11:40:38.658030 12082 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:40:38 dist-test-slave-tc2s krb5kdc[11011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189238, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:40:38.661303 12082 init.cc:377] Logged in from keytab as kudu/127.24.76.66@KRBTEST.COM (short username kudu)
I20260808 11:40:38.663048 12082 webserver.cc:460] Webserver started at http://127.24.76.66:42835/ using document root <none> and password file <none>
I20260808 11:40:38.663740 12082 fs_manager.cc:359] Metadata directory not provided
I20260808 11:40:38.663868 12082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:40:38.664219 12082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 11:40:38 dist-test-slave-tc2s krb5kdc[11011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189238, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for HTTP/127.24.76.84@KRBTEST.COM
I20260808 11:40:38.687575 12082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-1/data/instance:
uuid: "18ae2a9edc404e27ab727aaf2d84b5ea"
format_stamp: "Formatted at 2026-08-08 11:40:38 on dist-test-slave-tc2s"
server_key: "05f32acc42f68a8ba958b5f2d47fbb01"
server_key_iv: "96c67d4dd7f8511d499570fda543b955"
server_key_version: "kuduclusterkey@0"
I20260808 11:40:38.688844 12082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance:
uuid: "18ae2a9edc404e27ab727aaf2d84b5ea"
format_stamp: "Formatted at 2026-08-08 11:40:38 on dist-test-slave-tc2s"
server_key: "05f32acc42f68a8ba958b5f2d47fbb01"
server_key_iv: "96c67d4dd7f8511d499570fda543b955"
server_key_version: "kuduclusterkey@0"
I20260808 11:40:38.708251 12082 fs_manager.cc:691] Time spent creating directory manager: real 0.019s user 0.006s sys 0.011s
I20260808 11:40:38.758569 12125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:38.760764 12082 fs_manager.cc:725] Time spent opening block manager: real 0.020s user 0.008s sys 0.011s
I20260808 11:40:38.761080 12082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-1/wal
uuid: "18ae2a9edc404e27ab727aaf2d84b5ea"
format_stamp: "Formatted at 2026-08-08 11:40:38 on dist-test-slave-tc2s"
server_key: "05f32acc42f68a8ba958b5f2d47fbb01"
server_key_iv: "96c67d4dd7f8511d499570fda543b955"
server_key_version: "kuduclusterkey@0"
I20260808 11:40:38.761479 12082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:40:38.820416 12082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:40:38.826025 12082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:40:38.826438 12082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:40:38.842789 12082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:40:38.854874 12082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:40:38.855114 12082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260808 11:40:38.855350 12082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:40:38.855479 12082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:38.925676 12082 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.66:36803
I20260808 11:40:38.925704 12308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.66:36803 every 8 connection(s)
I20260808 11:40:38.927356 12082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
I20260808 11:40:38.932643 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 12082
I20260808 11:40:38.932832 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance
I20260808 11:40:38.933408 24881 external_mini_cluster.cc:1428] Setting key 2fd900e668dca0a183729fd8fe55912b
Aug 08 11:40:38 dist-test-slave-tc2s krb5kdc[11011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189238, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.76.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.67@KRBTEST.COM" created.
I20260808 11:40:38.974380 12315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:38.930100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:45891 (local address 127.24.76.126:45843)
0808 11:40:38.950289 (+ 20189us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:38.950295 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:38.950325 (+ 30us) server_negotiation.cc:407] Connection header received
0808 11:40:38.950426 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:38.950431 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:38.950522 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:38.950645 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:38.958085 (+ 7440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:38.959272 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:38.962288 (+ 3016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:38.962926 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:38.963105 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:38.967431 (+ 4326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:38.967439 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:38.967444 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:38.967560 (+ 116us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:38.970874 (+ 3314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:38.971706 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:38.971710 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:38.971712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:38.971909 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:38.972413 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:38.972417 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:38.972420 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:38.972812 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:38.972957 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:38.973727 (+ 770us) server_negotiation.cc:299] Negotiation successful
0808 11:40:38.974014 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20057,"thread_start_us":19934,"threads_started":1}
I20260808 11:40:38.981307 12317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:38.930710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:45843 (local address 127.24.76.66:45891)
0808 11:40:38.943048 (+ 12338us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:38.943102 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:38.945364 (+ 2262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:38.950918 (+ 5554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:38.950927 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:38.951403 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:40:38.952215 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:38.952242 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:38.959465 (+ 7223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:38.959472 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:38.962113 (+ 2641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:38.962136 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:38.963210 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:38.963219 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:38.963504 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:38.964280 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:40:38.964310 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:40:38.967144 (+ 2834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:40:38.971021 (+ 3877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:38.971029 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:38.971044 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:38.971574 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:38.972039 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:38.972044 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:38.972047 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:38.972256 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:38.972952 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:40:38.972961 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:40:38.973328 (+ 367us) client_negotiation.cc:769] Sending connection context
0808 11:40:38.973577 (+ 249us) client_negotiation.cc:240] Negotiation successful
0808 11:40:38.974147 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12026,"thread_start_us":11080,"threads_started":1}
I20260808 11:40:38.983292 12312 heartbeater.cc:347] Connected to a master server at 127.24.76.126:45843
I20260808 11:40:38.983613 12312 heartbeater.cc:464] Registering TS with master...
I20260808 11:40:38.984467 12312 heartbeater.cc:511] Master 127.24.76.126:45843 requested a full tablet report, sending...
I20260808 11:40:38.986140 11733 ts_manager.cc:194] Registered new tserver with Master: 18ae2a9edc404e27ab727aaf2d84b5ea (127.24.76.66:36803)
I20260808 11:40:38.987212 11733 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.66@KRBTEST.COM'} at 127.24.76.66:45891
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab.
Entry for principal kudu/127.24.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab.
I20260808 11:40:39.008487 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.76.84:40693/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.67:0
--local_ip_for_outbound_sockets=127.24.76.67
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:45843
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:39149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:40:39.092517 12076 heartbeater.cc:503] Master 127.24.76.126:45843 was elected leader, sending a full tablet report...
W20260808 11:40:39.451174 12320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:40:39.451483 12320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:40:39.451627 12320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:40:39.459668 12320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:40:39.459792 12320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:40:39.459939 12320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.67
I20260808 11:40:39.471961 12320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:39149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.24.76.84:40693/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:45843
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.67
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:40:39.473256 12320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:40:39.474639 12320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:40:39.491780 12325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:39.491719 12326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:39.492996 12328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:40:39.494216 12320 server_base.cc:1034] running on GCE node
I20260808 11:40:39.494845 12320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:40:39.495836 12320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:40:39.497087 12320 hybrid_clock.cc:648] HybridClock initialized: now 1786189239496981 us; error 51 us; skew 500 ppm
I20260808 11:40:39.497453 12320 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:40:39 dist-test-slave-tc2s krb5kdc[11011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189239, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:40:39.501295 12320 init.cc:377] Logged in from keytab as kudu/127.24.76.67@KRBTEST.COM (short username kudu)
I20260808 11:40:39.503309 12320 webserver.cc:460] Webserver started at http://127.24.76.67:35629/ using document root <none> and password file <none>
I20260808 11:40:39.504009 12320 fs_manager.cc:359] Metadata directory not provided
I20260808 11:40:39.504166 12320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:40:39.504542 12320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 11:40:39 dist-test-slave-tc2s krb5kdc[11011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189239, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for HTTP/127.24.76.84@KRBTEST.COM
I20260808 11:40:39.523321 12320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-2/data/instance:
uuid: "ee9bba657135494b82f36ed1aea03b84"
format_stamp: "Formatted at 2026-08-08 11:40:39 on dist-test-slave-tc2s"
server_key: "1d3cfa57c0ef82bcd8b88673ce9a7950"
server_key_iv: "4cb0174371d96ff57116ea9ee70a4da8"
server_key_version: "kuduclusterkey@0"
I20260808 11:40:39.524626 12320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance:
uuid: "ee9bba657135494b82f36ed1aea03b84"
format_stamp: "Formatted at 2026-08-08 11:40:39 on dist-test-slave-tc2s"
server_key: "1d3cfa57c0ef82bcd8b88673ce9a7950"
server_key_iv: "4cb0174371d96ff57116ea9ee70a4da8"
server_key_version: "kuduclusterkey@0"
I20260808 11:40:39.532806 12320 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.000s
I20260808 11:40:39.558748 12341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:39.560714 12320 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.006s sys 0.005s
I20260808 11:40:39.560916 12320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-2/wal
uuid: "ee9bba657135494b82f36ed1aea03b84"
format_stamp: "Formatted at 2026-08-08 11:40:39 on dist-test-slave-tc2s"
server_key: "1d3cfa57c0ef82bcd8b88673ce9a7950"
server_key_iv: "4cb0174371d96ff57116ea9ee70a4da8"
server_key_version: "kuduclusterkey@0"
I20260808 11:40:39.561193 12320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:40:39.610001 12320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:40:39.614925 12320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:40:39.615228 12320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:40:39.616472 12320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:40:39.618362 12320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:40:39.618449 12320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:39.618634 12320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:40:39.618718 12320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:39.673655 12320 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.67:45831
I20260808 11:40:39.673659 12523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.67:45831 every 8 connection(s)
I20260808 11:40:39.675547 12320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
I20260808 11:40:39.683878 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 12320
I20260808 11:40:39.684073 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance
I20260808 11:40:39.684752 24881 external_mini_cluster.cc:1428] Setting key 3716d07deac5a896f292ac59e4b0537a
Aug 08 11:40:39 dist-test-slave-tc2s krb5kdc[11011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189239, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:40:39.699505 12526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:39.678876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:54743 (local address 127.24.76.126:45843)
0808 11:40:39.679465 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:39.679471 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:39.681038 (+ 1567us) server_negotiation.cc:407] Connection header received
0808 11:40:39.683931 (+ 2893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:39.683935 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:39.684025 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:39.684131 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:39.686161 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:39.687133 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:39.689188 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:39.689863 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:39.690042 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:39.694015 (+ 3973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:39.694020 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:39.694024 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:39.694060 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:39.696636 (+ 2576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:39.697314 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:39.697317 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:39.697319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:39.697437 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:39.697820 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:39.697823 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:39.697825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:39.698096 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:39.698200 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:39.698871 (+ 671us) server_negotiation.cc:299] Negotiation successful
0808 11:40:39.699094 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":312,"threads_started":1}
I20260808 11:40:39.700515 12527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:39.679623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:45843 (local address 127.24.76.67:54743)
0808 11:40:39.680853 (+ 1230us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:39.680908 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:39.683625 (+ 2717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:39.684398 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:39.684410 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:39.685214 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:40:39.685974 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:39.686011 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:39.687288 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:39.687294 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:39.689053 (+ 1759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:39.689069 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:39.690175 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:39.690181 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:39.690373 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:39.691052 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:40:39.691085 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:40:39.693793 (+ 2708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:40:39.696759 (+ 2966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:39.696765 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:39.696776 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:39.697229 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:39.697514 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:39.697517 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:39.697519 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:39.697743 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:39.698199 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:40:39.698205 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:40:39.698508 (+ 303us) client_negotiation.cc:769] Sending connection context
0808 11:40:39.698780 (+ 272us) client_negotiation.cc:240] Negotiation successful
0808 11:40:39.699255 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":867,"thread_start_us":579,"threads_started":1}
I20260808 11:40:39.701798 12524 heartbeater.cc:347] Connected to a master server at 127.24.76.126:45843
I20260808 11:40:39.702096 12524 heartbeater.cc:464] Registering TS with master...
I20260808 11:40:39.702858 12524 heartbeater.cc:511] Master 127.24.76.126:45843 requested a full tablet report, sending...
I20260808 11:40:39.704237 11733 ts_manager.cc:194] Registered new tserver with Master: ee9bba657135494b82f36ed1aea03b84 (127.24.76.67:45831)
I20260808 11:40:39.705000 11733 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.67@KRBTEST.COM'} at 127.24.76.67:54743
I20260808 11:40:39.712698 24881 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:40:39.714370 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 11850
Aug 08 11:40:39 dist-test-slave-tc2s krb5kdc[11011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.65@KRBTEST.COM
I20260808 11:40:39.733178 12531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:39.715327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37534 (local address 127.24.76.65:44891)
0808 11:40:39.716127 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:39.716134 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:39.716165 (+ 31us) server_negotiation.cc:407] Connection header received
0808 11:40:39.716235 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:39.716244 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:39.716430 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:39.716571 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:39.718231 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:39.719522 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:39.721267 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:39.721865 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:39.722036 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:39.726966 (+ 4930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:39.726977 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:39.726998 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:39.727060 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:39.730152 (+ 3092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:39.730821 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:39.730828 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:39.730830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:39.731072 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:39.731448 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:39.731452 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:39.731454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:39.731803 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:39.732136 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:39.732385 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 11:40:39.732748 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":335,"threads_started":1}
I20260808 11:40:39.734426 12067 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:40:39.817103 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 11850
I20260808 11:40:39.841886 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 12082
Aug 08 11:40:39 dist-test-slave-tc2s krb5kdc[11011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.66@KRBTEST.COM
I20260808 11:40:39.858016 12533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:39.842874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48810 (local address 127.24.76.66:36803)
0808 11:40:39.843442 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:39.843450 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:39.843476 (+ 26us) server_negotiation.cc:407] Connection header received
0808 11:40:39.843550 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:39.843556 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:39.843780 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:39.843911 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:39.845057 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:39.846116 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:39.847497 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:39.848043 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:39.848138 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:39.852345 (+ 4207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:39.852357 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:39.852371 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:39.852412 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:39.855021 (+ 2609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:39.855803 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:39.855810 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:39.855812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:39.855950 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:39.856345 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:39.856349 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:39.856351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:39.856711 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:39.857008 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:39.857373 (+ 365us) server_negotiation.cc:299] Negotiation successful
0808 11:40:39.857653 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":315,"threads_started":1}
I20260808 11:40:39.859195 12303 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:40:39.939996 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 12082
I20260808 11:40:39.963364 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 12320
Aug 08 11:40:39 dist-test-slave-tc2s krb5kdc[11011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.67@KRBTEST.COM
I20260808 11:40:39.979496 12535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:39.964460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58140 (local address 127.24.76.67:45831)
0808 11:40:39.965160 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:39.965167 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:39.965197 (+ 30us) server_negotiation.cc:407] Connection header received
0808 11:40:39.965258 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:39.965265 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:39.965466 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:39.965607 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:39.966931 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:39.968056 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:39.969334 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:39.969869 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:39.970029 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:39.974309 (+ 4280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:39.974320 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:39.974334 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:39.974374 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:39.977121 (+ 2747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:39.977685 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:39.977692 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:39.977694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:39.977825 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:39.978149 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:39.978153 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:39.978155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:39.978491 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:39.978724 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:39.978942 (+ 218us) server_negotiation.cc:299] Negotiation successful
0808 11:40:39.979156 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":362,"threads_started":1}
I20260808 11:40:39.980758 12516 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:40:40.062038 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 12320
I20260808 11:40:40.085327 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 11686
I20260808 11:40:40.086591 11774 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:40:40.192265 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 11686
I20260808 11:40:40.218714 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:45843
--webserver_interface=127.24.76.126
--webserver_port=36805
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:39149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:45843
--ranger_config_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.76.84:40693/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.76.84:40693/kms with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:40:40.544241 12539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:40:40.544520 12539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:40:40.544653 12539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:40:40.551744 12539 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:40:40.551851 12539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:40:40.551906 12539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:40:40.551982 12539 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:40:40.552050 12539 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:40:40.561825 12539 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:39149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.76.84:40693/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:45843
--ranger_config_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:45843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=36805
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:40:40.563019 12539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:40:40.564298 12539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:40:40.575139 12544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:40.575248 12545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:40.575278 12547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:40:40.576134 12539 server_base.cc:1034] running on GCE node
I20260808 11:40:40.576856 12539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:40:40.578148 12539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:40:40.579340 12539 hybrid_clock.cc:648] HybridClock initialized: now 1786189240579292 us; error 44 us; skew 500 ppm
I20260808 11:40:40.579684 12539 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:40:40 dist-test-slave-tc2s krb5kdc[11011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189240, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:40:40.582647 12539 init.cc:377] Logged in from keytab as kudu/127.24.76.126@KRBTEST.COM (short username kudu)
I20260808 11:40:40.583935 12539 webserver.cc:460] Webserver started at http://127.24.76.126:36805/ using document root <none> and password file <none>
I20260808 11:40:40.584496 12539 fs_manager.cc:359] Metadata directory not provided
I20260808 11:40:40.584599 12539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 11:40:40 dist-test-slave-tc2s krb5kdc[11011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189240, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for HTTP/127.24.76.84@KRBTEST.COM
I20260808 11:40:40.627897 12539 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 11:40:40.630760 12562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:40.632050 12539 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:40:40.632190 12539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "5297282530a04795bf73dafd3716fee2"
format_stamp: "Formatted at 2026-08-08 11:40:36 on dist-test-slave-tc2s"
server_key: "f32d5c1e5f91e53d2212f4462b1a0705"
server_key_iv: "b9330846aa967b5f74ccacdd4f209c9e"
server_key_version: "kuduclusterkey@0"
I20260808 11:40:40.633160 12539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:40:40.648525 12539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:40:40.652205 12539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:40:40.652428 12539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:40:40.669263 12539 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:45843
I20260808 11:40:40.669241 12645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:45843 every 8 connection(s)
I20260808 11:40:40.670707 12539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:40:40.673528 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 12539
I20260808 11:40:40.674402 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.76.84:40693/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.65:44891
--local_ip_for_outbound_sockets=127.24.76.65
--tserver_master_addrs=127.24.76.126:45843
--webserver_port=34381
--webserver_interface=127.24.76.65
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:39149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.76.84:40693/kms with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:40:40.677371 12646 sys_catalog.cc:263] Verifying existing consensus state
I20260808 11:40:40.680550 12646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2: Bootstrap starting.
I20260808 11:40:40.698693 12646 log.cc:824] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2: Log is configured to *not* fsync() on all Append() calls
I20260808 11:40:40.707686 12646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 11:40:40.708181 12646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2: Bootstrap complete.
I20260808 11:40:40.718185 12646 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5297282530a04795bf73dafd3716fee2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 45843 } }
I20260808 11:40:40.719024 12646 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5297282530a04795bf73dafd3716fee2, State: Initialized, Role: FOLLOWER
I20260808 11:40:40.719594 12646 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5297282530a04795bf73dafd3716fee2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 45843 } }
I20260808 11:40:40.719810 12646 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:40:40.719924 12646 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:40:40.720090 12646 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2 [term 1 FOLLOWER]: Advancing to term 2
I20260808 11:40:40.723815 12646 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5297282530a04795bf73dafd3716fee2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 45843 } }
I20260808 11:40:40.724275 12646 leader_election.cc:302] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5297282530a04795bf73dafd3716fee2; no voters:
I20260808 11:40:40.724866 12646 leader_election.cc:288] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 11:40:40.725056 12650 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 11:40:40.725991 12650 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2 [term 2 LEADER]: Becoming Leader. State: Replica: 5297282530a04795bf73dafd3716fee2, State: Running, Role: LEADER
I20260808 11:40:40.726770 12646 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:40:40.726624 12650 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5297282530a04795bf73dafd3716fee2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 45843 } }
I20260808 11:40:40.729099 12653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5297282530a04795bf73dafd3716fee2. Latest consensus state: current_term: 2 leader_uuid: "5297282530a04795bf73dafd3716fee2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5297282530a04795bf73dafd3716fee2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 45843 } } }
I20260808 11:40:40.729588 12653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2 [sys.catalog]: This master's current role is: LEADER
I20260808 11:40:40.730360 12652 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5297282530a04795bf73dafd3716fee2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5297282530a04795bf73dafd3716fee2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 45843 } } }
I20260808 11:40:40.730669 12652 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2 [sys.catalog]: This master's current role is: LEADER
I20260808 11:40:40.736753 12660 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:40:40.742506 12660 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:40:40.743902 12660 catalog_manager.cc:1259] Loaded cluster ID: c5d5b0dd58bf4f13a427358dde354270
I20260808 11:40:40.744052 12660 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:40:40.747942 12660 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:40:40.749372 12660 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2: Loaded TSK: 0
I20260808 11:40:40.750926 12660 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:40:40.755570 12646 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260808 11:40:41.086488 12648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:40:41.086973 12648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:40:41.087198 12648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:40:41.098184 12648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:40:41.103084 12648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:40:41.103348 12648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.65
I20260808 11:40:41.118681 12648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:39149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.76.84:40693/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.65:44891
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.76.65
--webserver_port=34381
--tserver_master_addrs=127.24.76.126:45843
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.65
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:40:41.120628 12648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:40:41.122668 12648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:40:41.150709 12709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:40:41.153049 12648 server_base.cc:1034] running on GCE node
W20260808 11:40:41.153203 12712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:41.153609 12706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:40:41.154309 12648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:40:41.155416 12648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:40:41.156672 12648 hybrid_clock.cc:648] HybridClock initialized: now 1786189241156611 us; error 52 us; skew 500 ppm
I20260808 11:40:41.157014 12648 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:40:41 dist-test-slave-tc2s krb5kdc[11011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189241, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:40:41.161114 12648 init.cc:377] Logged in from keytab as kudu/127.24.76.65@KRBTEST.COM (short username kudu)
I20260808 11:40:41.168156 12648 webserver.cc:460] Webserver started at http://127.24.76.65:34381/ using document root <none> and password file <none>
I20260808 11:40:41.168905 12648 fs_manager.cc:359] Metadata directory not provided
I20260808 11:40:41.169130 12648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 11:40:41 dist-test-slave-tc2s krb5kdc[11011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189241, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for HTTP/127.24.76.84@KRBTEST.COM
I20260808 11:40:41.196480 12648 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 11:40:41.203744 12735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:41.205127 12648 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.006s sys 0.000s
I20260808 11:40:41.205359 12648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-0/wal
uuid: "2f089cd9b5f44ef981c66348b68e211f"
format_stamp: "Formatted at 2026-08-08 11:40:37 on dist-test-slave-tc2s"
server_key: "28f9394820d54ff894f0d833796fcd29"
server_key_iv: "2644780f0ce4be6cdb139e699e01b18f"
server_key_version: "kuduclusterkey@0"
I20260808 11:40:41.206516 12648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:40:41.239406 12648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:40:41.244563 12648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:40:41.244957 12648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:40:41.252755 12648 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:40:41.266732 12648 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:40:41.266928 12648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:41.267127 12648 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:40:41.267275 12648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:41.340900 12648 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.65:44891
I20260808 11:40:41.340955 12918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.65:44891 every 8 connection(s)
I20260808 11:40:41.342427 12648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
I20260808 11:40:41.351418 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 12648
I20260808 11:40:41.352250 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.76.84:40693/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.66:36803
--local_ip_for_outbound_sockets=127.24.76.66
--tserver_master_addrs=127.24.76.126:45843
--webserver_port=42835
--webserver_interface=127.24.76.66
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:39149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.76.84:40693/kms with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 11:40:41 dist-test-slave-tc2s krb5kdc[11011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189241, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:40:41.376914 12926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:41.345772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:45843 (local address 127.24.76.65:36929)
0808 11:40:41.350699 (+ 4927us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:41.350743 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:41.353113 (+ 2370us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:41.355503 (+ 2390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:41.355512 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:41.356059 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:40:41.356780 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:41.356815 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:41.359281 (+ 2466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:41.359287 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:41.361274 (+ 1987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:41.361291 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:41.362865 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:41.362873 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:41.363117 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:41.363878 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:40:41.363905 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:40:41.366333 (+ 2428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:40:41.371834 (+ 5501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:41.371844 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:41.371860 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:41.372470 (+ 610us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:41.372989 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:41.372994 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:41.372998 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:41.373241 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:41.374249 (+ 1008us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:40:41.374257 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:40:41.374662 (+ 405us) client_negotiation.cc:769] Sending connection context
0808 11:40:41.374975 (+ 313us) client_negotiation.cc:240] Negotiation successful
0808 11:40:41.375529 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4705,"thread_start_us":3931,"threads_started":1}
I20260808 11:40:41.376938 12925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:41.345792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:36929 (local address 127.24.76.126:45843)
0808 11:40:41.349666 (+ 3874us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:41.349685 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:41.351281 (+ 1596us) server_negotiation.cc:407] Connection header received
0808 11:40:41.353882 (+ 2601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:41.353927 (+ 45us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:41.354401 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:41.354780 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:41.357039 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:41.359067 (+ 2028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:41.361497 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:41.362614 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:41.362808 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:41.366626 (+ 3818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:41.366634 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:41.366647 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:41.366692 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:41.371652 (+ 4960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:41.372635 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:41.372644 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:41.372651 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:41.372856 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:41.373397 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:41.373401 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:41.373405 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:41.374066 (+ 661us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:41.374275 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:41.375251 (+ 976us) server_negotiation.cc:299] Negotiation successful
0808 11:40:41.375695 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3605,"thread_start_us":3359,"threads_started":1}
I20260808 11:40:41.380690 12919 heartbeater.cc:347] Connected to a master server at 127.24.76.126:45843
I20260808 11:40:41.381017 12919 heartbeater.cc:464] Registering TS with master...
I20260808 11:40:41.381937 12919 heartbeater.cc:511] Master 127.24.76.126:45843 requested a full tablet report, sending...
W20260808 11:40:41.385231 12919 heartbeater.cc:644] Failed to heartbeat to 127.24.76.126:45843 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 11:40:41.385483 12919 heartbeater.cc:464] Registering TS with master...
I20260808 11:40:41.385761 12919 heartbeater.cc:511] Master 127.24.76.126:45843 requested a full tablet report, sending...
I20260808 11:40:41.387027 12919 heartbeater.cc:464] Registering TS with master...
I20260808 11:40:41.387311 12919 heartbeater.cc:511] Master 127.24.76.126:45843 requested a full tablet report, sending...
W20260808 11:40:41.388449 12919 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 11:40:41.745116 12927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:40:41.745398 12927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:40:41.745563 12927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:40:41.752918 12927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:40:41.753034 12927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:40:41.753211 12927 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.66
I20260808 11:40:41.763438 12927 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:39149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.76.84:40693/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.66:36803
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.76.66
--webserver_port=42835
--tserver_master_addrs=127.24.76.126:45843
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.66
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:40:41.764674 12927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:40:41.765969 12927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:40:41.780380 12947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:41.780334 12948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:41.780308 12953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:40:41.782251 12927 server_base.cc:1034] running on GCE node
I20260808 11:40:41.782836 12927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:40:41.783816 12927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:40:41.785121 12927 hybrid_clock.cc:648] HybridClock initialized: now 1786189241785056 us; error 53 us; skew 500 ppm
I20260808 11:40:41.785440 12927 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:40:41 dist-test-slave-tc2s krb5kdc[11011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189241, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:40:41.788919 12927 init.cc:377] Logged in from keytab as kudu/127.24.76.66@KRBTEST.COM (short username kudu)
I20260808 11:40:41.791092 12927 webserver.cc:460] Webserver started at http://127.24.76.66:42835/ using document root <none> and password file <none>
I20260808 11:40:41.792045 12927 fs_manager.cc:359] Metadata directory not provided
I20260808 11:40:41.792208 12927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 11:40:41 dist-test-slave-tc2s krb5kdc[11011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189241, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for HTTP/127.24.76.84@KRBTEST.COM
I20260808 11:40:41.813020 12927 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 11:40:41.816027 12974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:41.817517 12927 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 11:40:41.817678 12927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-1/wal
uuid: "18ae2a9edc404e27ab727aaf2d84b5ea"
format_stamp: "Formatted at 2026-08-08 11:40:38 on dist-test-slave-tc2s"
server_key: "05f32acc42f68a8ba958b5f2d47fbb01"
server_key_iv: "96c67d4dd7f8511d499570fda543b955"
server_key_version: "kuduclusterkey@0"
I20260808 11:40:41.818763 12927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:40:41.848628 12927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:40:41.854053 12927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:40:41.854529 12927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:40:41.866345 12927 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:40:41.878749 12927 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:40:41.878942 12927 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:41.879365 12927 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:40:41.881019 12927 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260808 11:40:41.949283 12927 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.66:36803
I20260808 11:40:41.949347 13151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.66:36803 every 8 connection(s)
I20260808 11:40:41.950913 12927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
I20260808 11:40:41.961355 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 12927
I20260808 11:40:41.962173 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.76.84:40693/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.67:45831
--local_ip_for_outbound_sockets=127.24.76.67
--tserver_master_addrs=127.24.76.126:45843
--webserver_port=35629
--webserver_interface=127.24.76.67
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:39149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.76.84:40693/kms with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 11:40:41 dist-test-slave-tc2s krb5kdc[11011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189241, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
Aug 08 11:40:41 dist-test-slave-tc2s krb5kdc[11011](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786189241, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:40:41.980978 13157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:41.953960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:38023 (local address 127.24.76.126:45843)
0808 11:40:41.959996 (+ 6036us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:41.960002 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:41.960025 (+ 23us) server_negotiation.cc:407] Connection header received
0808 11:40:41.961468 (+ 1443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:41.961474 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:41.961573 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:41.961686 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:41.963862 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:41.964939 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:41.967292 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:41.967918 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:41.968043 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:41.973246 (+ 5203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:41.973253 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:41.973257 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:41.973300 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:41.977072 (+ 3772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:41.978037 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:41.978043 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:41.978047 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:41.978211 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:41.978782 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:41.978787 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:41.978791 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:41.979198 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:41.979331 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:41.980274 (+ 943us) server_negotiation.cc:299] Negotiation successful
0808 11:40:41.980550 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5901,"thread_start_us":5784,"threads_started":1}
I20260808 11:40:41.983429 13158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:41.954641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:45843 (local address 127.24.76.66:38023)
0808 11:40:41.958504 (+ 3863us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:41.958550 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:41.960978 (+ 2428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:41.962155 (+ 1177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:41.962168 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:41.962818 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:40:41.963637 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:41.963665 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:41.965135 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:41.965141 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:41.967112 (+ 1971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:41.967129 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:41.968372 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:41.968381 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:41.968683 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:41.969457 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:40:41.969496 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:40:41.972247 (+ 2751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:40:41.977247 (+ 5000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:41.977258 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:41.977275 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:41.977852 (+ 577us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:41.978363 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:41.978368 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:41.978371 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:41.978645 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:41.979433 (+ 788us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:40:41.979441 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:40:41.979840 (+ 399us) client_negotiation.cc:769] Sending connection context
0808 11:40:41.981424 (+ 1584us) client_negotiation.cc:240] Negotiation successful
0808 11:40:41.982045 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3557,"thread_start_us":3335,"threads_started":1}
I20260808 11:40:41.985335 13154 heartbeater.cc:347] Connected to a master server at 127.24.76.126:45843
I20260808 11:40:41.985653 13154 heartbeater.cc:464] Registering TS with master...
I20260808 11:40:41.986558 13154 heartbeater.cc:511] Master 127.24.76.126:45843 requested a full tablet report, sending...
I20260808 11:40:41.988596 12589 ts_manager.cc:194] Registered new tserver with Master: 18ae2a9edc404e27ab727aaf2d84b5ea (127.24.76.66:36803)
I20260808 11:40:41.990904 12589 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.66@KRBTEST.COM'} at 127.24.76.66:38023
I20260808 11:40:42.388605 12919 heartbeater.cc:464] Registering TS with master...
I20260808 11:40:42.389088 12919 heartbeater.cc:511] Master 127.24.76.126:45843 requested a full tablet report, sending...
I20260808 11:40:42.390589 12589 ts_manager.cc:194] Registered new tserver with Master: 2f089cd9b5f44ef981c66348b68e211f (127.24.76.65:44891)
I20260808 11:40:42.391947 12589 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.65@KRBTEST.COM'} at 127.24.76.65:36929
Aug 08 11:40:42 dist-test-slave-tc2s krb5kdc[11011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189241, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for HTTP/127.24.76.84@KRBTEST.COM
W20260808 11:40:42.469950 13161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:40:42.470244 13161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:40:42.470414 13161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:40:42.477860 13161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:40:42.477999 13161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:40:42.478156 13161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.67
I20260808 11:40:42.487977 13161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:39149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.76.84:40693/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.67:45831
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.76.67
--webserver_port=35629
--tserver_master_addrs=127.24.76.126:45843
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.67
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:40:42.489271 13161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:40:42.490581 13161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:40:42.505831 13172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:42.505950 13171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:40:42.517822 13161 server_base.cc:1034] running on GCE node
W20260808 11:40:42.518134 13174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:40:42.518985 13161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:40:42.520296 13161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:40:42.521529 13161 hybrid_clock.cc:648] HybridClock initialized: now 1786189242521466 us; error 47 us; skew 500 ppm
I20260808 11:40:42.521944 13161 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:40:42 dist-test-slave-tc2s krb5kdc[11011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189242, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:40:42.526566 13161 init.cc:377] Logged in from keytab as kudu/127.24.76.67@KRBTEST.COM (short username kudu)
I20260808 11:40:42.531329 13161 webserver.cc:460] Webserver started at http://127.24.76.67:35629/ using document root <none> and password file <none>
I20260808 11:40:42.532279 13161 fs_manager.cc:359] Metadata directory not provided
I20260808 11:40:42.532431 13161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 11:40:42 dist-test-slave-tc2s krb5kdc[11011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189242, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for HTTP/127.24.76.84@KRBTEST.COM
I20260808 11:40:42.556370 13161 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 11:40:42.560091 13187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:42.561883 13161 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260808 11:40:42.562110 13161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-2/wal
uuid: "ee9bba657135494b82f36ed1aea03b84"
format_stamp: "Formatted at 2026-08-08 11:40:39 on dist-test-slave-tc2s"
server_key: "1d3cfa57c0ef82bcd8b88673ce9a7950"
server_key_iv: "4cb0174371d96ff57116ea9ee70a4da8"
server_key_version: "kuduclusterkey@0"
I20260808 11:40:42.563618 13161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:40:42.586009 13161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:40:42.591166 13161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:40:42.591485 13161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:40:42.592825 13161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:40:42.594677 13161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:40:42.594789 13161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:42.594905 13161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:40:42.594986 13161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:42.653975 13161 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.67:45831
I20260808 11:40:42.654018 13364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.67:45831 every 8 connection(s)
I20260808 11:40:42.655545 13161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
I20260808 11:40:42.665158 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 13161
Aug 08 11:40:42 dist-test-slave-tc2s krb5kdc[11011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189242, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:40:42.698041 13367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:42.662499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:45843 (local address 127.24.76.67:50653)
0808 11:40:42.672773 (+ 10274us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:42.672835 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:42.675473 (+ 2638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:42.677739 (+ 2266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:42.677752 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:42.678415 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:40:42.679269 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:42.679295 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:42.680919 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:42.680925 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:42.684787 (+ 3862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:42.684804 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:42.686046 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:42.686054 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:42.686353 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:42.687132 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:40:42.687155 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:40:42.689482 (+ 2327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:40:42.693203 (+ 3721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:42.693211 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:42.693228 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:42.693856 (+ 628us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:42.694318 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:42.694322 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:42.694325 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:42.694553 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:42.695257 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:40:42.695265 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:40:42.695663 (+ 398us) client_negotiation.cc:769] Sending connection context
0808 11:40:42.695978 (+ 315us) client_negotiation.cc:240] Negotiation successful
0808 11:40:42.696530 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9932,"thread_start_us":13265,"threads_started":1}
I20260808 11:40:42.705372 13368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:42.663048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:50653 (local address 127.24.76.126:45843)
0808 11:40:42.675446 (+ 12398us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:42.675453 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:42.675476 (+ 23us) server_negotiation.cc:407] Connection header received
0808 11:40:42.675954 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:42.675959 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:42.676066 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:42.676172 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:42.679503 (+ 3331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:42.680708 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:42.684971 (+ 4263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:42.685608 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:42.685727 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:42.689756 (+ 4029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:42.689763 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:42.689768 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:42.689822 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:42.693068 (+ 3246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:42.694027 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:42.694033 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:42.694037 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:42.694183 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:42.694682 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:42.694688 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:42.694691 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:42.695044 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:42.695172 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:42.704119 (+ 8947us) server_negotiation.cc:299] Negotiation successful
0808 11:40:42.704354 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12270,"thread_start_us":12434,"threads_started":1}
I20260808 11:40:42.706710 13365 heartbeater.cc:347] Connected to a master server at 127.24.76.126:45843
I20260808 11:40:42.707062 13365 heartbeater.cc:464] Registering TS with master...
I20260808 11:40:42.708047 13365 heartbeater.cc:511] Master 127.24.76.126:45843 requested a full tablet report, sending...
I20260808 11:40:42.709744 12589 ts_manager.cc:194] Registered new tserver with Master: ee9bba657135494b82f36ed1aea03b84 (127.24.76.67:45831)
I20260808 11:40:42.710897 12589 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.67@KRBTEST.COM'} at 127.24.76.67:50653
I20260808 11:40:42.712698 13371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:42.675639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41032 (local address 127.24.76.126:45843)
0808 11:40:42.682720 (+ 7081us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:42.682726 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:42.682743 (+ 17us) server_negotiation.cc:407] Connection header received
0808 11:40:42.682791 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:42.682796 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:42.682894 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:42.683009 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:42.686828 (+ 3819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:42.691638 (+ 4810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:42.695132 (+ 3494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:42.695781 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:42.699139 (+ 3358us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:42.699410 (+ 271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:42.699416 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:42.699421 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:42.699452 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:42.702914 (+ 3462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:42.703757 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:42.703762 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:42.703765 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:42.703912 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:42.704419 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:42.704424 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:42.704428 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:42.708487 (+ 4059us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:42.708760 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:42.711149 (+ 2389us) server_negotiation.cc:299] Negotiation successful
0808 11:40:42.712366 (+ 1217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6965,"thread_start_us":7513,"threads_started":1}
I20260808 11:40:42.716779 24881 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 11:40:42 dist-test-slave-tc2s krb5kdc[11011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189241, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for HTTP/127.24.76.84@KRBTEST.COM
I20260808 11:40:42.793110 13371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:42.745684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41040 (local address 127.24.76.126:45843)
0808 11:40:42.745935 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:42.745942 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:42.770525 (+ 24583us) server_negotiation.cc:407] Connection header received
0808 11:40:42.770850 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:42.770857 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:42.770948 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:42.771022 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:42.773317 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:42.774487 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:42.776591 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:42.777232 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:42.777375 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:42.786867 (+ 9492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:42.786875 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:42.786880 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:42.786930 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:42.789703 (+ 2773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:42.790568 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:42.790574 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:42.790577 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:42.790748 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:42.791244 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:42.791250 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:42.791254 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:42.791656 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:42.791898 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:42.792503 (+ 605us) server_negotiation.cc:299] Negotiation successful
0808 11:40:42.792755 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 11:40:42.800098 12589 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41040:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:40:42.812358 12589 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:40:42.877547 13391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:42.839416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:44891 (local address 127.0.0.1:51284)
0808 11:40:42.867719 (+ 28303us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:42.867739 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:42.867877 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:42.868946 (+ 1069us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:42.868955 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:42.868988 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:40:42.869478 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:42.869500 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:42.871950 (+ 2450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:42.871956 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:42.874347 (+ 2391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:42.874364 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:42.876585 (+ 2221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:42.876591 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:42.876791 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:42.876847 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 11:40:42.876983 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 11:40:42.877171 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28169,"thread_start_us":21992,"threads_started":1}
I20260808 11:40:42.878523 13394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:42.839757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51284 (local address 127.24.76.65:44891)
0808 11:40:42.861178 (+ 21421us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:42.861187 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:42.868020 (+ 6833us) server_negotiation.cc:407] Connection header received
0808 11:40:42.868140 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:42.868147 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:42.868394 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:42.868659 (+ 265us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:40:42.869820 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:42.871764 (+ 1944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:42.874543 (+ 2779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:42.876017 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:42.876124 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:42.876377 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:42.877981 (+ 1604us) server_negotiation.cc:299] Negotiation successful
0808 11:40:42.878164 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20975,"thread_start_us":21173,"threads_started":1}
I20260808 11:40:42.884047 12813 tablet_service.cc:1467] Processing CreateTablet for tablet e650875560914c5c84bef89a601d5bd5 (DEFAULT_TABLE table=test-table [id=f4aa7e2c2144481ea1ddac62732bc2a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:40:42.885818 12813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e650875560914c5c84bef89a601d5bd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:40:42.893404 13391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:42.875038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:36803 (local address 127.0.0.1:52744)
0808 11:40:42.880478 (+ 5440us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:42.880496 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:42.880593 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:42.886757 (+ 6164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:42.886762 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:42.886788 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:40:42.886969 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:42.886981 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:42.889001 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:42.889006 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:42.890815 (+ 1809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:42.890847 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:42.892616 (+ 1769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:42.892623 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:42.892814 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:42.892852 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 11:40:42.892928 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 11:40:42.893095 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5340,"threads_started":1}
I20260808 11:40:42.894443 13395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:42.868648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52744 (local address 127.24.76.66:36803)
0808 11:40:42.886163 (+ 17515us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:42.886171 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:42.886200 (+ 29us) server_negotiation.cc:407] Connection header received
0808 11:40:42.886305 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:42.886312 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:42.886532 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:42.887158 (+ 626us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:40:42.887403 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:42.888831 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:42.890970 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:42.892409 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:42.893622 (+ 1213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:42.893820 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:42.893933 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 11:40:42.894102 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17355,"thread_start_us":15756,"threads_started":1}
I20260808 11:40:42.896178 13393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:42.837826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:45831 (local address 127.0.0.1:51048)
0808 11:40:42.866153 (+ 28327us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:42.866222 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:42.866483 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:42.877980 (+ 11497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:42.877986 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:42.878018 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:40:42.878409 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:42.878420 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:42.891078 (+ 12658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:42.891083 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:42.893053 (+ 1970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:42.893070 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:42.895473 (+ 2403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:42.895479 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:42.895640 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:42.895670 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 11:40:42.895744 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 11:40:42.895873 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27986,"thread_start_us":29605,"threads_started":1}
I20260808 11:40:42.897059 13392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:42.838542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51048 (local address 127.24.76.67:45831)
0808 11:40:42.854524 (+ 15982us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:42.854535 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:42.866401 (+ 11866us) server_negotiation.cc:407] Connection header received
0808 11:40:42.866735 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:42.866745 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:42.867059 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:42.867291 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:40:42.878639 (+ 11348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:42.880564 (+ 1925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:42.893217 (+ 12653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:42.894702 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:42.894831 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:42.895049 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:42.896562 (+ 1513us) server_negotiation.cc:299] Negotiation successful
0808 11:40:42.896734 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15732,"thread_start_us":15537,"threads_started":1}
I20260808 11:40:42.901204 13046 tablet_service.cc:1467] Processing CreateTablet for tablet e650875560914c5c84bef89a601d5bd5 (DEFAULT_TABLE table=test-table [id=f4aa7e2c2144481ea1ddac62732bc2a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:40:42.902700 13046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e650875560914c5c84bef89a601d5bd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:40:42.902237 13262 tablet_service.cc:1467] Processing CreateTablet for tablet e650875560914c5c84bef89a601d5bd5 (DEFAULT_TABLE table=test-table [id=f4aa7e2c2144481ea1ddac62732bc2a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:40:42.903641 13262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e650875560914c5c84bef89a601d5bd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:40:42.929519 13398 tablet_bootstrap.cc:492] T e650875560914c5c84bef89a601d5bd5 P 2f089cd9b5f44ef981c66348b68e211f: Bootstrap starting.
I20260808 11:40:42.933230 13399 tablet_bootstrap.cc:492] T e650875560914c5c84bef89a601d5bd5 P 18ae2a9edc404e27ab727aaf2d84b5ea: Bootstrap starting.
I20260808 11:40:42.936934 13397 tablet_bootstrap.cc:492] T e650875560914c5c84bef89a601d5bd5 P ee9bba657135494b82f36ed1aea03b84: Bootstrap starting.
I20260808 11:40:42.940446 13398 tablet_bootstrap.cc:654] T e650875560914c5c84bef89a601d5bd5 P 2f089cd9b5f44ef981c66348b68e211f: Neither blocks nor log segments found. Creating new log.
I20260808 11:40:42.941813 13398 log.cc:824] T e650875560914c5c84bef89a601d5bd5 P 2f089cd9b5f44ef981c66348b68e211f: Log is configured to *not* fsync() on all Append() calls
I20260808 11:40:42.952189 13399 tablet_bootstrap.cc:654] T e650875560914c5c84bef89a601d5bd5 P 18ae2a9edc404e27ab727aaf2d84b5ea: Neither blocks nor log segments found. Creating new log.
I20260808 11:40:42.954367 13397 tablet_bootstrap.cc:654] T e650875560914c5c84bef89a601d5bd5 P ee9bba657135494b82f36ed1aea03b84: Neither blocks nor log segments found. Creating new log.
I20260808 11:40:42.955271 13398 tablet_bootstrap.cc:492] T e650875560914c5c84bef89a601d5bd5 P 2f089cd9b5f44ef981c66348b68e211f: No bootstrap required, opened a new log
I20260808 11:40:42.955638 13398 ts_tablet_manager.cc:1397] T e650875560914c5c84bef89a601d5bd5 P 2f089cd9b5f44ef981c66348b68e211f: Time spent bootstrapping tablet: real 0.026s user 0.009s sys 0.005s
I20260808 11:40:42.955649 13397 log.cc:824] T e650875560914c5c84bef89a601d5bd5 P ee9bba657135494b82f36ed1aea03b84: Log is configured to *not* fsync() on all Append() calls
I20260808 11:40:42.964968 13397 tablet_bootstrap.cc:492] T e650875560914c5c84bef89a601d5bd5 P ee9bba657135494b82f36ed1aea03b84: No bootstrap required, opened a new log
I20260808 11:40:42.965333 13397 ts_tablet_manager.cc:1397] T e650875560914c5c84bef89a601d5bd5 P ee9bba657135494b82f36ed1aea03b84: Time spent bootstrapping tablet: real 0.029s user 0.008s sys 0.000s
I20260808 11:40:42.967160 13399 log.cc:824] T e650875560914c5c84bef89a601d5bd5 P 18ae2a9edc404e27ab727aaf2d84b5ea: Log is configured to *not* fsync() on all Append() calls
I20260808 11:40:42.967444 13398 raft_consensus.cc:348] T e650875560914c5c84bef89a601d5bd5 P 2f089cd9b5f44ef981c66348b68e211f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bba657135494b82f36ed1aea03b84" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 45831 } } peers { permanent_uuid: "2f089cd9b5f44ef981c66348b68e211f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44891 } } peers { permanent_uuid: "18ae2a9edc404e27ab727aaf2d84b5ea" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 36803 } }
I20260808 11:40:42.968008 13398 raft_consensus.cc:374] T e650875560914c5c84bef89a601d5bd5 P 2f089cd9b5f44ef981c66348b68e211f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:40:42.968158 13398 raft_consensus.cc:729] T e650875560914c5c84bef89a601d5bd5 P 2f089cd9b5f44ef981c66348b68e211f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f089cd9b5f44ef981c66348b68e211f, State: Initialized, Role: FOLLOWER
I20260808 11:40:42.968971 13398 consensus_queue.cc:260] T e650875560914c5c84bef89a601d5bd5 P 2f089cd9b5f44ef981c66348b68e211f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bba657135494b82f36ed1aea03b84" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 45831 } } peers { permanent_uuid: "2f089cd9b5f44ef981c66348b68e211f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44891 } } peers { permanent_uuid: "18ae2a9edc404e27ab727aaf2d84b5ea" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 36803 } }
I20260808 11:40:42.980399 13397 raft_consensus.cc:348] T e650875560914c5c84bef89a601d5bd5 P ee9bba657135494b82f36ed1aea03b84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bba657135494b82f36ed1aea03b84" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 45831 } } peers { permanent_uuid: "2f089cd9b5f44ef981c66348b68e211f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44891 } } peers { permanent_uuid: "18ae2a9edc404e27ab727aaf2d84b5ea" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 36803 } }
I20260808 11:40:42.991024 13399 tablet_bootstrap.cc:492] T e650875560914c5c84bef89a601d5bd5 P 18ae2a9edc404e27ab727aaf2d84b5ea: No bootstrap required, opened a new log
I20260808 11:40:42.993290 13399 ts_tablet_manager.cc:1397] T e650875560914c5c84bef89a601d5bd5 P 18ae2a9edc404e27ab727aaf2d84b5ea: Time spent bootstrapping tablet: real 0.067s user 0.007s sys 0.009s
I20260808 11:40:42.994683 12919 heartbeater.cc:503] Master 127.24.76.126:45843 was elected leader, sending a full tablet report...
I20260808 11:40:42.995450 13398 ts_tablet_manager.cc:1428] T e650875560914c5c84bef89a601d5bd5 P 2f089cd9b5f44ef981c66348b68e211f: Time spent starting tablet: real 0.040s user 0.021s sys 0.015s
I20260808 11:40:42.995927 13397 raft_consensus.cc:374] T e650875560914c5c84bef89a601d5bd5 P ee9bba657135494b82f36ed1aea03b84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:40:42.996088 13397 raft_consensus.cc:729] T e650875560914c5c84bef89a601d5bd5 P ee9bba657135494b82f36ed1aea03b84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee9bba657135494b82f36ed1aea03b84, State: Initialized, Role: FOLLOWER
I20260808 11:40:42.996678 13154 heartbeater.cc:503] Master 127.24.76.126:45843 was elected leader, sending a full tablet report...
I20260808 11:40:42.996903 13397 consensus_queue.cc:260] T e650875560914c5c84bef89a601d5bd5 P ee9bba657135494b82f36ed1aea03b84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bba657135494b82f36ed1aea03b84" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 45831 } } peers { permanent_uuid: "2f089cd9b5f44ef981c66348b68e211f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44891 } } peers { permanent_uuid: "18ae2a9edc404e27ab727aaf2d84b5ea" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 36803 } }
I20260808 11:40:43.002038 13397 ts_tablet_manager.cc:1428] T e650875560914c5c84bef89a601d5bd5 P ee9bba657135494b82f36ed1aea03b84: Time spent starting tablet: real 0.037s user 0.007s sys 0.008s
I20260808 11:40:43.002151 13365 heartbeater.cc:503] Master 127.24.76.126:45843 was elected leader, sending a full tablet report...
I20260808 11:40:43.003753 13399 raft_consensus.cc:348] T e650875560914c5c84bef89a601d5bd5 P 18ae2a9edc404e27ab727aaf2d84b5ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bba657135494b82f36ed1aea03b84" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 45831 } } peers { permanent_uuid: "2f089cd9b5f44ef981c66348b68e211f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44891 } } peers { permanent_uuid: "18ae2a9edc404e27ab727aaf2d84b5ea" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 36803 } }
I20260808 11:40:43.004103 13399 raft_consensus.cc:374] T e650875560914c5c84bef89a601d5bd5 P 18ae2a9edc404e27ab727aaf2d84b5ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:40:43.004205 13399 raft_consensus.cc:729] T e650875560914c5c84bef89a601d5bd5 P 18ae2a9edc404e27ab727aaf2d84b5ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18ae2a9edc404e27ab727aaf2d84b5ea, State: Initialized, Role: FOLLOWER
I20260808 11:40:43.004782 13399 consensus_queue.cc:260] T e650875560914c5c84bef89a601d5bd5 P 18ae2a9edc404e27ab727aaf2d84b5ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bba657135494b82f36ed1aea03b84" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 45831 } } peers { permanent_uuid: "2f089cd9b5f44ef981c66348b68e211f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44891 } } peers { permanent_uuid: "18ae2a9edc404e27ab727aaf2d84b5ea" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 36803 } }
I20260808 11:40:43.007601 13399 ts_tablet_manager.cc:1428] T e650875560914c5c84bef89a601d5bd5 P 18ae2a9edc404e27ab727aaf2d84b5ea: Time spent starting tablet: real 0.014s user 0.005s sys 0.007s
W20260808 11:40:43.095125 12923 tablet.cc:2369] T e650875560914c5c84bef89a601d5bd5 P 2f089cd9b5f44ef981c66348b68e211f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:40:43.163283 13366 tablet.cc:2369] T e650875560914c5c84bef89a601d5bd5 P ee9bba657135494b82f36ed1aea03b84: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:40:43.211830 13156 tablet.cc:2369] T e650875560914c5c84bef89a601d5bd5 P 18ae2a9edc404e27ab727aaf2d84b5ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:40:43.305495 13411 raft_consensus.cc:482] T e650875560914c5c84bef89a601d5bd5 P 18ae2a9edc404e27ab727aaf2d84b5ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:40:43.305809 13411 raft_consensus.cc:504] T e650875560914c5c84bef89a601d5bd5 P 18ae2a9edc404e27ab727aaf2d84b5ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bba657135494b82f36ed1aea03b84" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 45831 } } peers { permanent_uuid: "2f089cd9b5f44ef981c66348b68e211f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44891 } } peers { permanent_uuid: "18ae2a9edc404e27ab727aaf2d84b5ea" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 36803 } }
I20260808 11:40:43.307669 13411 leader_election.cc:288] T e650875560914c5c84bef89a601d5bd5 P 18ae2a9edc404e27ab727aaf2d84b5ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee9bba657135494b82f36ed1aea03b84 (127.24.76.67:45831), 2f089cd9b5f44ef981c66348b68e211f (127.24.76.65:44891)
I20260808 11:40:43.307883 13404 raft_consensus.cc:482] T e650875560914c5c84bef89a601d5bd5 P 2f089cd9b5f44ef981c66348b68e211f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:40:43.308218 13404 raft_consensus.cc:504] T e650875560914c5c84bef89a601d5bd5 P 2f089cd9b5f44ef981c66348b68e211f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bba657135494b82f36ed1aea03b84" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 45831 } } peers { permanent_uuid: "2f089cd9b5f44ef981c66348b68e211f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44891 } } peers { permanent_uuid: "18ae2a9edc404e27ab727aaf2d84b5ea" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 36803 } }
I20260808 11:40:43.310243 13404 leader_election.cc:288] T e650875560914c5c84bef89a601d5bd5 P 2f089cd9b5f44ef981c66348b68e211f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee9bba657135494b82f36ed1aea03b84 (127.24.76.67:45831), 18ae2a9edc404e27ab727aaf2d84b5ea (127.24.76.66:36803)
I20260808 11:40:43.317037 13394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:43.308276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:37847 (local address 127.24.76.65:44891)
0808 11:40:43.308503 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:43.308510 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:43.309440 (+ 930us) server_negotiation.cc:407] Connection header received
0808 11:40:43.309691 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:43.309696 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:43.309775 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:43.309874 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:40:43.310523 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:43.311652 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:43.314549 (+ 2897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:43.315864 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:43.316000 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:43.316074 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:43.316626 (+ 552us) server_negotiation.cc:299] Negotiation successful
0808 11:40:43.316757 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 11:40:43.317348 13392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:43.308225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:50823 (local address 127.24.76.67:45831)
0808 11:40:43.308431 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:43.308439 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:43.309118 (+ 679us) server_negotiation.cc:407] Connection header received
0808 11:40:43.309489 (+ 371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:43.309495 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:43.309574 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:43.309634 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:40:43.310529 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:43.311662 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:43.314989 (+ 3327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:43.316216 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:43.316332 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:43.316404 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:43.316993 (+ 589us) server_negotiation.cc:299] Negotiation successful
0808 11:40:43.317130 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 11:40:43.317760 13427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:43.308033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:44891 (local address 127.24.76.66:37847)
0808 11:40:43.309352 (+ 1319us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:43.309368 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:43.309530 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:43.309922 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:43.309926 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:43.309969 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:40:43.310342 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:43.310354 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:43.311836 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:43.311840 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:43.314306 (+ 2466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:43.314324 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:43.316265 (+ 1941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:43.316272 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:43.316433 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:43.316465 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 11:40:43.317255 (+ 790us) client_negotiation.cc:240] Negotiation successful
0808 11:40:43.317385 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1210,"mutex_wait_us":489,"thread_start_us":508,"threads_started":1}
I20260808 11:40:43.318791 12848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e650875560914c5c84bef89a601d5bd5" candidate_uuid: "18ae2a9edc404e27ab727aaf2d84b5ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f089cd9b5f44ef981c66348b68e211f" is_pre_election: true
I20260808 11:40:43.319278 12848 raft_consensus.cc:2454] T e650875560914c5c84bef89a601d5bd5 P 2f089cd9b5f44ef981c66348b68e211f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18ae2a9edc404e27ab727aaf2d84b5ea in term 0.
I20260808 11:40:43.320133 13426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:43.308033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:45831 (local address 127.24.76.66:50823)
0808 11:40:43.308992 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:43.309016 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:43.309260 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:43.309799 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:43.309802 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:43.309823 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:40:43.310351 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:43.310360 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:43.311833 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:43.311838 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:43.314810 (+ 2972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:43.314825 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:43.316599 (+ 1774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:43.316604 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:43.316815 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:43.316849 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 11:40:43.318167 (+ 1318us) client_negotiation.cc:240] Negotiation successful
0808 11:40:43.319721 (+ 1554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":824,"mutex_wait_us":258,"thread_start_us":606,"threads_started":1}
I20260808 11:40:43.320035 12977 leader_election.cc:302] T e650875560914c5c84bef89a601d5bd5 P 18ae2a9edc404e27ab727aaf2d84b5ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18ae2a9edc404e27ab727aaf2d84b5ea, 2f089cd9b5f44ef981c66348b68e211f; no voters:
I20260808 11:40:43.320701 13411 raft_consensus.cc:2781] T e650875560914c5c84bef89a601d5bd5 P 18ae2a9edc404e27ab727aaf2d84b5ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:40:43.320830 13411 raft_consensus.cc:482] T e650875560914c5c84bef89a601d5bd5 P 18ae2a9edc404e27ab727aaf2d84b5ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:40:43.320924 13411 raft_consensus.cc:3037] T e650875560914c5c84bef89a601d5bd5 P 18ae2a9edc404e27ab727aaf2d84b5ea [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:40:43.324098 13411 raft_consensus.cc:504] T e650875560914c5c84bef89a601d5bd5 P 18ae2a9edc404e27ab727aaf2d84b5ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bba657135494b82f36ed1aea03b84" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 45831 } } peers { permanent_uuid: "2f089cd9b5f44ef981c66348b68e211f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44891 } } peers { permanent_uuid: "18ae2a9edc404e27ab727aaf2d84b5ea" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 36803 } }
I20260808 11:40:43.324836 13411 leader_election.cc:288] T e650875560914c5c84bef89a601d5bd5 P 18ae2a9edc404e27ab727aaf2d84b5ea [CANDIDATE]: Term 1 election: Requested vote from peers ee9bba657135494b82f36ed1aea03b84 (127.24.76.67:45831), 2f089cd9b5f44ef981c66348b68e211f (127.24.76.65:44891)
I20260808 11:40:43.324937 13392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:43.310667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:32979 (local address 127.24.76.67:45831)
0808 11:40:43.317571 (+ 6904us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:43.317577 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:43.317594 (+ 17us) server_negotiation.cc:407] Connection header received
0808 11:40:43.317648 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:43.317653 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:43.317723 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:43.317825 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:40:43.318814 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:43.319845 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:43.322313 (+ 2468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:43.323598 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:43.323737 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:43.323820 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:43.324457 (+ 637us) server_negotiation.cc:299] Negotiation successful
0808 11:40:43.324578 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6810,"thread_start_us":13654,"threads_started":1}
I20260808 11:40:43.325572 13428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:43.310415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:45831 (local address 127.24.76.65:32979)
0808 11:40:43.312393 (+ 1978us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:43.312412 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:43.312534 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:43.318145 (+ 5611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:43.318149 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:43.318179 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:40:43.318642 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:43.318655 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:43.320021 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:43.320026 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:43.322138 (+ 2112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:43.322152 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:43.324059 (+ 1907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:43.324066 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:43.324264 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:43.324296 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 11:40:43.325103 (+ 807us) client_negotiation.cc:240] Negotiation successful
0808 11:40:43.325233 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1853,"thread_start_us":1571,"threads_started":1}
I20260808 11:40:43.325543 12848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e650875560914c5c84bef89a601d5bd5" candidate_uuid: "18ae2a9edc404e27ab727aaf2d84b5ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f089cd9b5f44ef981c66348b68e211f"
I20260808 11:40:43.326365 12848 raft_consensus.cc:3037] T e650875560914c5c84bef89a601d5bd5 P 2f089cd9b5f44ef981c66348b68e211f [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:40:43.326392 13289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e650875560914c5c84bef89a601d5bd5" candidate_uuid: "18ae2a9edc404e27ab727aaf2d84b5ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee9bba657135494b82f36ed1aea03b84"
I20260808 11:40:43.326421 13288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e650875560914c5c84bef89a601d5bd5" candidate_uuid: "2f089cd9b5f44ef981c66348b68e211f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee9bba657135494b82f36ed1aea03b84" is_pre_election: true
I20260808 11:40:43.326892 13289 raft_consensus.cc:3037] T e650875560914c5c84bef89a601d5bd5 P ee9bba657135494b82f36ed1aea03b84 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:40:43.328096 13290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e650875560914c5c84bef89a601d5bd5" candidate_uuid: "18ae2a9edc404e27ab727aaf2d84b5ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee9bba657135494b82f36ed1aea03b84" is_pre_election: true
I20260808 11:40:43.329619 12848 raft_consensus.cc:2454] T e650875560914c5c84bef89a601d5bd5 P 2f089cd9b5f44ef981c66348b68e211f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18ae2a9edc404e27ab727aaf2d84b5ea in term 1.
I20260808 11:40:43.330054 12977 leader_election.cc:302] T e650875560914c5c84bef89a601d5bd5 P 18ae2a9edc404e27ab727aaf2d84b5ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18ae2a9edc404e27ab727aaf2d84b5ea, 2f089cd9b5f44ef981c66348b68e211f; no voters:
I20260808 11:40:43.330119 13289 raft_consensus.cc:2454] T e650875560914c5c84bef89a601d5bd5 P ee9bba657135494b82f36ed1aea03b84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18ae2a9edc404e27ab727aaf2d84b5ea in term 1.
I20260808 11:40:43.330189 13434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:43.312479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:36803 (local address 127.24.76.65:33699)
0808 11:40:43.323010 (+ 10531us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:43.323029 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:43.323340 (+ 311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:43.323818 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:43.323823 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:43.323848 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:40:43.324285 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:43.324295 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:43.325495 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:43.325500 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:43.327688 (+ 2188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:43.327702 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:43.329421 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:43.329428 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:43.329592 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:43.329627 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 11:40:43.329712 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 11:40:43.329823 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10356,"thread_start_us":10625,"threads_started":1}
I20260808 11:40:43.330199 13395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:43.312876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:33699 (local address 127.24.76.66:36803)
0808 11:40:43.313039 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:43.313045 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:43.323112 (+ 10067us) server_negotiation.cc:407] Connection header received
0808 11:40:43.323510 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:43.323516 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:43.323586 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:43.323644 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:40:43.324461 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:43.325343 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:43.327870 (+ 2527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:43.329015 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:43.329152 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:43.329219 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:43.329766 (+ 547us) server_negotiation.cc:299] Negotiation successful
0808 11:40:43.329911 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 11:40:43.330725 13411 raft_consensus.cc:2781] T e650875560914c5c84bef89a601d5bd5 P 18ae2a9edc404e27ab727aaf2d84b5ea [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:40:43.331212 13411 raft_consensus.cc:686] T e650875560914c5c84bef89a601d5bd5 P 18ae2a9edc404e27ab727aaf2d84b5ea [term 1 LEADER]: Becoming Leader. State: Replica: 18ae2a9edc404e27ab727aaf2d84b5ea, State: Running, Role: LEADER
I20260808 11:40:43.330991 13077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e650875560914c5c84bef89a601d5bd5" candidate_uuid: "2f089cd9b5f44ef981c66348b68e211f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18ae2a9edc404e27ab727aaf2d84b5ea" is_pre_election: true
I20260808 11:40:43.331789 13411 consensus_queue.cc:237] T e650875560914c5c84bef89a601d5bd5 P 18ae2a9edc404e27ab727aaf2d84b5ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bba657135494b82f36ed1aea03b84" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 45831 } } peers { permanent_uuid: "2f089cd9b5f44ef981c66348b68e211f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44891 } } peers { permanent_uuid: "18ae2a9edc404e27ab727aaf2d84b5ea" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 36803 } }
I20260808 11:40:43.332013 12738 leader_election.cc:302] T e650875560914c5c84bef89a601d5bd5 P 2f089cd9b5f44ef981c66348b68e211f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2f089cd9b5f44ef981c66348b68e211f; no voters: 18ae2a9edc404e27ab727aaf2d84b5ea, ee9bba657135494b82f36ed1aea03b84
I20260808 11:40:43.332386 13404 raft_consensus.cc:2726] T e650875560914c5c84bef89a601d5bd5 P 2f089cd9b5f44ef981c66348b68e211f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 11:40:43.336275 12585 catalog_manager.cc:5495] T e650875560914c5c84bef89a601d5bd5 P 18ae2a9edc404e27ab727aaf2d84b5ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 18ae2a9edc404e27ab727aaf2d84b5ea (127.24.76.66). New cstate: current_term: 1 leader_uuid: "18ae2a9edc404e27ab727aaf2d84b5ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9bba657135494b82f36ed1aea03b84" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 45831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f089cd9b5f44ef981c66348b68e211f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18ae2a9edc404e27ab727aaf2d84b5ea" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 36803 } health_report { overall_health: HEALTHY } } }
I20260808 11:40:43.367120 13395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:43.361846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52746 (local address 127.24.76.66:36803)
0808 11:40:43.362014 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:43.362018 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:43.362389 (+ 371us) server_negotiation.cc:407] Connection header received
0808 11:40:43.362568 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:43.362572 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:43.362666 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:43.362786 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:40:43.363095 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:43.363871 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:43.365200 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:43.365615 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:43.365700 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:43.366190 (+ 490us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:40:43.366680 (+ 490us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:40:43.366779 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:43.366835 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 11:40:43.366897 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 11:40:43.374632 13289 raft_consensus.cc:1261] T e650875560914c5c84bef89a601d5bd5 P ee9bba657135494b82f36ed1aea03b84 [term 1 FOLLOWER]: Refusing update from remote peer 18ae2a9edc404e27ab727aaf2d84b5ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:40:43.374666 12848 raft_consensus.cc:1261] T e650875560914c5c84bef89a601d5bd5 P 2f089cd9b5f44ef981c66348b68e211f [term 1 FOLLOWER]: Refusing update from remote peer 18ae2a9edc404e27ab727aaf2d84b5ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:40:43.375439 13435 consensus_queue.cc:1035] T e650875560914c5c84bef89a601d5bd5 P 18ae2a9edc404e27ab727aaf2d84b5ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9bba657135494b82f36ed1aea03b84" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 45831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:40:43.375746 13411 consensus_queue.cc:1035] T e650875560914c5c84bef89a601d5bd5 P 18ae2a9edc404e27ab727aaf2d84b5ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f089cd9b5f44ef981c66348b68e211f" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:40:43.400946 12589 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41040:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:40:43.401238 12589 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41040:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:40:43.404991 12589 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5297282530a04795bf73dafd3716fee2: Sending DeleteTablet for 3 replicas of tablet e650875560914c5c84bef89a601d5bd5
I20260808 11:40:43.406176 12813 tablet_service.cc:1514] Processing DeleteTablet for tablet e650875560914c5c84bef89a601d5bd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:40:43 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:51284
I20260808 11:40:43.406983 13262 tablet_service.cc:1514] Processing DeleteTablet for tablet e650875560914c5c84bef89a601d5bd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:40:43 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:51048
I20260808 11:40:43.407423 13046 tablet_service.cc:1514] Processing DeleteTablet for tablet e650875560914c5c84bef89a601d5bd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:40:43 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:52744
I20260808 11:40:43.407929 13462 tablet_replica.cc:330] T e650875560914c5c84bef89a601d5bd5 P 2f089cd9b5f44ef981c66348b68e211f: stopping tablet replica
I20260808 11:40:43.408304 13462 raft_consensus.cc:2229] T e650875560914c5c84bef89a601d5bd5 P 2f089cd9b5f44ef981c66348b68e211f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:40:43.408835 13462 raft_consensus.cc:2258] T e650875560914c5c84bef89a601d5bd5 P 2f089cd9b5f44ef981c66348b68e211f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:40:43.410970 13462 ts_tablet_manager.cc:1905] T e650875560914c5c84bef89a601d5bd5 P 2f089cd9b5f44ef981c66348b68e211f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:40:43.413463 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 12648
I20260808 11:40:43.418251 13463 tablet_replica.cc:330] T e650875560914c5c84bef89a601d5bd5 P ee9bba657135494b82f36ed1aea03b84: stopping tablet replica
I20260808 11:40:43.418598 13463 raft_consensus.cc:2229] T e650875560914c5c84bef89a601d5bd5 P ee9bba657135494b82f36ed1aea03b84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:40:43.419005 13463 raft_consensus.cc:2258] T e650875560914c5c84bef89a601d5bd5 P ee9bba657135494b82f36ed1aea03b84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:40:43.419724 13464 tablet_replica.cc:330] T e650875560914c5c84bef89a601d5bd5 P 18ae2a9edc404e27ab727aaf2d84b5ea: stopping tablet replica
I20260808 11:40:43.420042 13464 raft_consensus.cc:2229] T e650875560914c5c84bef89a601d5bd5 P 18ae2a9edc404e27ab727aaf2d84b5ea [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:40:43.420557 13464 raft_consensus.cc:2258] T e650875560914c5c84bef89a601d5bd5 P 18ae2a9edc404e27ab727aaf2d84b5ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:40:43.422323 13464 ts_tablet_manager.cc:1905] T e650875560914c5c84bef89a601d5bd5 P 18ae2a9edc404e27ab727aaf2d84b5ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:40:43.427201 13463 ts_tablet_manager.cc:1905] T e650875560914c5c84bef89a601d5bd5 P ee9bba657135494b82f36ed1aea03b84: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:40:43.427822 13464 ts_tablet_manager.cc:1918] T e650875560914c5c84bef89a601d5bd5 P 18ae2a9edc404e27ab727aaf2d84b5ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:40:43.427963 13464 log.cc:1196] T e650875560914c5c84bef89a601d5bd5 P 18ae2a9edc404e27ab727aaf2d84b5ea: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-1/wal/wals/e650875560914c5c84bef89a601d5bd5
I20260808 11:40:43.428458 13464 ts_tablet_manager.cc:1939] T e650875560914c5c84bef89a601d5bd5 P 18ae2a9edc404e27ab727aaf2d84b5ea: Deleting consensus metadata
I20260808 11:40:43.430197 12565 catalog_manager.cc:4841] TS 18ae2a9edc404e27ab727aaf2d84b5ea (127.24.76.66:36803): tablet e650875560914c5c84bef89a601d5bd5 (table test-table [id=f4aa7e2c2144481ea1ddac62732bc2a4]) successfully deleted
I20260808 11:40:43.433871 13463 ts_tablet_manager.cc:1918] T e650875560914c5c84bef89a601d5bd5 P ee9bba657135494b82f36ed1aea03b84: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:40:43.434056 13463 log.cc:1196] T e650875560914c5c84bef89a601d5bd5 P ee9bba657135494b82f36ed1aea03b84: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-2/wal/wals/e650875560914c5c84bef89a601d5bd5
I20260808 11:40:43.434577 13463 ts_tablet_manager.cc:1939] T e650875560914c5c84bef89a601d5bd5 P ee9bba657135494b82f36ed1aea03b84: Deleting consensus metadata
I20260808 11:40:43.436281 12565 catalog_manager.cc:4841] TS ee9bba657135494b82f36ed1aea03b84 (127.24.76.67:45831): tablet e650875560914c5c84bef89a601d5bd5 (table test-table [id=f4aa7e2c2144481ea1ddac62732bc2a4]) successfully deleted
I20260808 11:40:43.437548 13462 ts_tablet_manager.cc:1918] T e650875560914c5c84bef89a601d5bd5 P 2f089cd9b5f44ef981c66348b68e211f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:40:43.437713 13462 log.cc:1196] T e650875560914c5c84bef89a601d5bd5 P 2f089cd9b5f44ef981c66348b68e211f: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786189042529715-24881-0/minicluster-data/ts-0/wal/wals/e650875560914c5c84bef89a601d5bd5
I20260808 11:40:43.438268 13462 ts_tablet_manager.cc:1939] T e650875560914c5c84bef89a601d5bd5 P 2f089cd9b5f44ef981c66348b68e211f: Deleting consensus metadata
I20260808 11:40:43.440064 12569 catalog_manager.cc:4841] TS 2f089cd9b5f44ef981c66348b68e211f (127.24.76.65:44891): tablet e650875560914c5c84bef89a601d5bd5 (table test-table [id=f4aa7e2c2144481ea1ddac62732bc2a4]) successfully deleted
I20260808 11:40:43.440515 13394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:43.414298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51296 (local address 127.24.76.65:44891)
0808 11:40:43.414464 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:43.414471 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:43.426373 (+ 11902us) server_negotiation.cc:407] Connection header received
0808 11:40:43.426568 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:43.426574 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:43.426654 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:43.426769 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:43.428534 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:43.429586 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:43.431319 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:43.431730 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:43.431854 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:43.434295 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:43.434311 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:43.434329 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:43.434369 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:43.437473 (+ 3104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:43.438286 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:43.438295 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:43.438297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:43.438469 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:43.438862 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:43.438867 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:43.438871 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:43.439270 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:43.439596 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:43.439926 (+ 330us) server_negotiation.cc:299] Negotiation successful
0808 11:40:43.440128 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 11:40:43.441267 12913 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:40:43.532259 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 12648
I20260808 11:40:43.554600 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 12927
I20260808 11:40:43.569959 13395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:43.555595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52762 (local address 127.24.76.66:36803)
0808 11:40:43.555825 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:43.555835 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:43.555856 (+ 21us) server_negotiation.cc:407] Connection header received
0808 11:40:43.555924 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:43.555930 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:43.556012 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:43.556127 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:43.557367 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:43.558516 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:43.559781 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:43.560453 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:43.560573 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:43.563210 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:43.563227 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:43.563248 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:43.563293 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:43.567068 (+ 3775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:43.567743 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:43.567752 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:43.567756 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:43.567943 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:43.568274 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:43.568279 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:43.568283 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:43.568819 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:43.569147 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:43.569331 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 11:40:43.569566 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 11:40:43.570709 13142 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:40:43.685555 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 12927
I20260808 11:40:43.711290 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 13161
I20260808 11:40:43.724635 13392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:43.712093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51052 (local address 127.24.76.67:45831)
0808 11:40:43.712275 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:43.712281 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:43.712301 (+ 20us) server_negotiation.cc:407] Connection header received
0808 11:40:43.712369 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:43.712373 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:43.712434 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:43.712517 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:43.713927 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:43.714692 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:43.716034 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:43.716450 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:43.716563 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:43.719129 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:43.719146 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:43.719164 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:43.719230 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:43.721823 (+ 2593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:43.722459 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:43.722469 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:43.722472 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:43.722638 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:43.723002 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:43.723008 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:43.723012 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:43.723426 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:43.723716 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:43.724015 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 11:40:43.724238 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 11:40:43.725507 13357 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:40:43.813376 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 13161
I20260808 11:40:43.836179 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 12539
I20260808 11:40:43.837423 12641 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:40:43.948071 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 12539
I20260808 11:40:43.985220 24881 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 11:40:44.329363 24881 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 11:40:44.329571 24881 mini_ranger.cc:66] Stopping Ranger...
I20260808 11:40:44.673525 24881 mini_ranger.cc:68] Stopped Ranger
2026-08-08 11:40:44.673 UTC [11051] LOG: received smart shutdown request
2026-08-08 11:40:44.676 UTC [11051] LOG: background worker "logical replication launcher" (PID 11067) exited with exit code 1
2026-08-08 11:40:44.676 UTC [11060] LOG: shutting down
2026-08-08 11:40:44.805 UTC [11051] LOG: database system is shut down
2026-08-08T11:40:44Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51095 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:40:45 dist-test-slave-tc2s krb5kdc[13483](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:40:45 dist-test-slave-tc2s krb5kdc[13483](info): set up 2 sockets
Aug 08 11:40:45 dist-test-slave-tc2s krb5kdc[13483](info): commencing operation
krb5kdc: starting...
W20260808 11:40:46.127559 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.001s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:40:46 dist-test-slave-tc2s krb5kdc[13483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189246, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:40:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:40:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
I20260808 11:40:46.375389 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:43431
--webserver_interface=127.24.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:39067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:43431,127.24.76.125:46461,127.24.76.124:43521
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:40:46.715451 13501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:40:46.715714 13501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:40:46.715818 13501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:40:46.722769 13501 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:40:46.722872 13501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:40:46.722924 13501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:40:46.722980 13501 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:40:46.723023 13501 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:40:46.732249 13501 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:39067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:43431,127.24.76.125:46461,127.24.76.124:43521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:43431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:40:46.733796 13501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:40:46.735306 13501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:40:46.746590 13513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:46.746676 13510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:46.746675 13511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:40:46.748080 13501 server_base.cc:1034] running on GCE node
I20260808 11:40:46.748834 13501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:40:46.750123 13501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:40:46.751331 13501 hybrid_clock.cc:648] HybridClock initialized: now 1786189246751281 us; error 38 us; skew 500 ppm
I20260808 11:40:46.751652 13501 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:40:46 dist-test-slave-tc2s krb5kdc[13483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189246, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:40:46.754551 13501 init.cc:377] Logged in from keytab as kudu/127.24.76.126@KRBTEST.COM (short username kudu)
I20260808 11:40:46.755944 13501 webserver.cc:460] Webserver started at http://127.24.76.126:37749/ using document root <none> and password file <none>
I20260808 11:40:46.756582 13501 fs_manager.cc:359] Metadata directory not provided
I20260808 11:40:46.756696 13501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:40:46.756986 13501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:40:46.759871 13501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "76572ad082cd49758fe05a17dfb0d686"
format_stamp: "Formatted at 2026-08-08 11:40:46 on dist-test-slave-tc2s"
server_key: "ed0c76115072fa101883fc5294a65d76"
server_key_iv: "8ab322c9b8cc27f92211ace49e5ff9c5"
server_key_version: "encryptionkey@0"
I20260808 11:40:46.760756 13501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "76572ad082cd49758fe05a17dfb0d686"
format_stamp: "Formatted at 2026-08-08 11:40:46 on dist-test-slave-tc2s"
server_key: "ed0c76115072fa101883fc5294a65d76"
server_key_iv: "8ab322c9b8cc27f92211ace49e5ff9c5"
server_key_version: "encryptionkey@0"
I20260808 11:40:46.765712 13501 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260808 11:40:46.768901 13521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:46.770231 13501 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:40:46.770385 13501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "76572ad082cd49758fe05a17dfb0d686"
format_stamp: "Formatted at 2026-08-08 11:40:46 on dist-test-slave-tc2s"
server_key: "ed0c76115072fa101883fc5294a65d76"
server_key_iv: "8ab322c9b8cc27f92211ace49e5ff9c5"
server_key_version: "encryptionkey@0"
I20260808 11:40:46.770553 13501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:40:46.784323 13501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:40:46.787977 13501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:40:46.788220 13501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:40:46.805330 13501 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:43431
I20260808 11:40:46.805361 13608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:43431 every 8 connection(s)
I20260808 11:40:46.806608 13501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:40:46.810232 13609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:40:46.816184 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 13501
I20260808 11:40:46.816428 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:40:46.817247 24881 external_mini_cluster.cc:1428] Setting key c7265c3b7a58d03a32a9d678be8c775c
I20260808 11:40:46.816797 13609 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.76.126" port: 43431 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:40:46 dist-test-slave-tc2s krb5kdc[13483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189246, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:40:46.839998 13613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:46.820400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32836 (local address 127.24.76.126:43431)
0808 11:40:46.822183 (+ 1783us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:46.822189 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:46.822233 (+ 44us) server_negotiation.cc:407] Connection header received
0808 11:40:46.824057 (+ 1824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:46.824062 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:46.824281 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:46.824432 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:46.825935 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:46.827130 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:46.828932 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:46.829482 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:46.829654 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:46.833079 (+ 3425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:46.833087 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:46.833100 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:46.833138 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:46.835792 (+ 2654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:46.836415 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:46.836421 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:46.836423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:46.836560 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:46.836937 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:46.836940 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:46.836942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:46.837442 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:46.837629 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:46.838217 (+ 588us) server_negotiation.cc:299] Negotiation successful
0808 11:40:46.838512 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1618,"spinlock_wait_cycles":34560,"thread_start_us":1389,"threads_started":1}
I20260808 11:40:46.840036 13612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:46.820376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:43431 (local address 127.0.0.1:32836)
0808 11:40:46.821228 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:46.821277 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:46.823694 (+ 2417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:46.824530 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:46.824537 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:46.825234 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:40:46.825689 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:46.825713 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:46.827286 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:46.827291 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:46.828780 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:46.828795 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:46.829730 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:46.829738 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:46.829920 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:46.830565 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:40:46.830586 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:40:46.832872 (+ 2286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:40:46.835910 (+ 3038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:46.835922 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:46.835934 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:46.836299 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:46.836663 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:46.836668 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:46.836672 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:46.836837 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:46.837574 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:40:46.837583 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:40:46.837800 (+ 217us) client_negotiation.cc:769] Sending connection context
0808 11:40:46.838076 (+ 276us) client_negotiation.cc:240] Negotiation successful
0808 11:40:46.838672 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"mutex_wait_us":128,"thread_start_us":351,"threads_started":1}
I20260808 11:40:46.843155 13609 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.76.125" port: 46461 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.24.76.125@KRBTEST.COM; defaulting to no policy
I20260808 11:40:46.844198 13612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:46.843810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:46461 (local address 127.0.0.1:39472)
0808 11:40:46.843985 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:46.844088 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.125:46461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260808 11:40:46.844440 13528 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.76.125:46461: connect: Connection refused (error 111)
I20260808 11:40:46.845834 13612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:46.845522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:46461 (local address 127.0.0.1:39484)
0808 11:40:46.845633 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:46.845699 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.125:46461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 11:40:46.846087 13609 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.125:46461: Network error: Client connection negotiation failed: client connection to 127.24.76.125:46461: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.125.keytab.
Entry for principal kudu/127.24.76.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.125.keytab.
I20260808 11:40:46.874716 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.125:46461
--webserver_interface=127.24.76.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.125.keytab
--principal=kudu/127.24.76.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:39067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:43431,127.24.76.125:46461,127.24.76.124:43521
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:40:46.895455 13609 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.125" port: 46461 } attempt: 1
I20260808 11:40:46.896580 13612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:46.896124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:46461 (local address 127.0.0.1:39500)
0808 11:40:46.896313 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:46.896421 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.125:46461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 11:40:46.897657 13612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:46.897317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:46461 (local address 127.0.0.1:39508)
0808 11:40:46.897458 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:46.897528 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.125:46461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 11:40:46.897859 13609 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.125:46461: Network error: Client connection negotiation failed: client connection to 127.24.76.125:46461: connect: Connection refused (error 111)
I20260808 11:40:46.966114 13609 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.125" port: 46461 } attempt: 2
I20260808 11:40:46.967311 13612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:46.966856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:46461 (local address 127.0.0.1:39516)
0808 11:40:46.967080 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:46.967170 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.125:46461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 11:40:46.968345 13612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:46.968005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:46461 (local address 127.0.0.1:39524)
0808 11:40:46.968135 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:46.968221 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.125:46461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 11:40:46.968524 13609 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.125:46461: Network error: Client connection negotiation failed: client connection to 127.24.76.125:46461: connect: Connection refused (error 111)
I20260808 11:40:47.059738 13609 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.125" port: 46461 } attempt: 3
I20260808 11:40:47.060909 13612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.060475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:46461 (local address 127.0.0.1:39530)
0808 11:40:47.060670 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:47.060775 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.125:46461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 11:40:47.062000 13612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.061584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:46461 (local address 127.0.0.1:39534)
0808 11:40:47.061734 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:47.061807 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.125:46461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 11:40:47.062198 13609 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.125:46461: Network error: Client connection negotiation failed: client connection to 127.24.76.125:46461: connect: Connection refused (error 111)
W20260808 11:40:47.201795 13619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:40:47.202078 13619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:40:47.202174 13619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 11:40:47.205404 13609 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.125" port: 46461 } attempt: 4
I20260808 11:40:47.206671 13612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.206146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:46461 (local address 127.0.0.1:39536)
0808 11:40:47.206437 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:47.206537 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.125:46461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157}
I20260808 11:40:47.207636 13612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.207293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:46461 (local address 127.0.0.1:39550)
0808 11:40:47.207413 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:47.207513 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.125:46461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 11:40:47.207806 13609 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.125:46461: Network error: Client connection negotiation failed: client connection to 127.24.76.125:46461: connect: Connection refused (error 111)
W20260808 11:40:47.209540 13619 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:40:47.209630 13619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:40:47.209682 13619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:40:47.209745 13619 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:40:47.209789 13619 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:40:47.219404 13619 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:39067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:43431,127.24.76.125:46461,127.24.76.124:43521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.125.keytab
--principal=kudu/127.24.76.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.125:46461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.76.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:40:47.220633 13619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:40:47.221884 13619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:40:47.232913 13634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:47.233050 13630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:47.233006 13631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:40:47.233863 13619 server_base.cc:1034] running on GCE node
I20260808 11:40:47.234591 13619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:40:47.235940 13619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:40:47.237144 13619 hybrid_clock.cc:648] HybridClock initialized: now 1786189247237099 us; error 38 us; skew 500 ppm
I20260808 11:40:47.237453 13619 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:40:47 dist-test-slave-tc2s krb5kdc[13483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189247, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:40:47.240406 13619 init.cc:377] Logged in from keytab as kudu/127.24.76.125@KRBTEST.COM (short username kudu)
I20260808 11:40:47.241719 13619 webserver.cc:460] Webserver started at http://127.24.76.125:34381/ using document root <none> and password file <none>
I20260808 11:40:47.242421 13619 fs_manager.cc:359] Metadata directory not provided
I20260808 11:40:47.242537 13619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:40:47.242810 13619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:40:47.245579 13619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-1/data/instance:
uuid: "5f62a83d9dc34126b38fca7d1f0413c3"
format_stamp: "Formatted at 2026-08-08 11:40:47 on dist-test-slave-tc2s"
server_key: "b66d40b804fced1d0322b6837a1e7010"
server_key_iv: "03dd47e748af2e905cfaf6e06c783eff"
server_key_version: "encryptionkey@0"
I20260808 11:40:47.246356 13619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-1/wal/instance:
uuid: "5f62a83d9dc34126b38fca7d1f0413c3"
format_stamp: "Formatted at 2026-08-08 11:40:47 on dist-test-slave-tc2s"
server_key: "b66d40b804fced1d0322b6837a1e7010"
server_key_iv: "03dd47e748af2e905cfaf6e06c783eff"
server_key_version: "encryptionkey@0"
I20260808 11:40:47.251034 13619 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 11:40:47.254225 13641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:47.255398 13619 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:40:47.255550 13619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-1/wal
uuid: "5f62a83d9dc34126b38fca7d1f0413c3"
format_stamp: "Formatted at 2026-08-08 11:40:47 on dist-test-slave-tc2s"
server_key: "b66d40b804fced1d0322b6837a1e7010"
server_key_iv: "03dd47e748af2e905cfaf6e06c783eff"
server_key_version: "encryptionkey@0"
I20260808 11:40:47.255713 13619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:40:47.292510 13619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:40:47.296219 13619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:40:47.296469 13619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:40:47.313258 13619 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.125:46461
I20260808 11:40:47.313292 13711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.125:46461 every 8 connection(s)
I20260808 11:40:47.314533 13619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-1/data/info.pb
I20260808 11:40:47.318035 13712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:40:47.318456 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 13619
I20260808 11:40:47.318674 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-1/wal/instance
I20260808 11:40:47.319247 24881 external_mini_cluster.cc:1428] Setting key 9c476a922ed6c73729089ca950345a3a
I20260808 11:40:47.326555 13712 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.76.126" port: 43431 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:40:47 dist-test-slave-tc2s krb5kdc[13483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189247, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.125@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.76.124@KRBTEST.COM; defaulting to no policy
I20260808 11:40:47.347036 13613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.329562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32844 (local address 127.24.76.126:43431)
0808 11:40:47.329762 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:47.329768 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:47.330867 (+ 1099us) server_negotiation.cc:407] Connection header received
0808 11:40:47.332786 (+ 1919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:47.332792 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:47.332875 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:47.332986 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:47.334408 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.335337 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.336790 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.337411 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.337530 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:47.340698 (+ 3168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:47.340703 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:47.340707 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:47.340745 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:47.344182 (+ 3437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:47.344866 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:47.344872 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:47.344876 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:47.345036 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:47.345400 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:47.345404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:47.345408 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:47.345765 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:47.345953 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:47.346576 (+ 623us) server_negotiation.cc:299] Negotiation successful
0808 11:40:47.346802 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 11:40:47.347903 13715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.329913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:43431 (local address 127.0.0.1:32844)
0808 11:40:47.330681 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:47.330735 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:47.332522 (+ 1787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:47.333161 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:47.333168 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:47.333646 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:40:47.334224 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:47.334242 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.335468 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.335471 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:47.336642 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:47.336654 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.337592 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.337596 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:47.337762 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:47.338392 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:40:47.338419 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:40:47.340482 (+ 2063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:40:47.344316 (+ 3834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:47.344323 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:47.344334 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:47.344741 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:47.345131 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:47.345134 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:47.345136 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:47.345291 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:47.345871 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:40:47.345876 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:40:47.346208 (+ 332us) client_negotiation.cc:769] Sending connection context
0808 11:40:47.346425 (+ 217us) client_negotiation.cc:240] Negotiation successful
0808 11:40:47.346795 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":289,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.124@KRBTEST.COM" created.
I20260808 11:40:47.349565 13712 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.76.125" port: 46461 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:40:47 dist-test-slave-tc2s krb5kdc[13483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189247, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.125@KRBTEST.COM for kudu/127.24.76.125@KRBTEST.COM
I20260808 11:40:47.363883 13715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.350321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:46461 (local address 127.0.0.1:39564)
0808 11:40:47.350480 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:47.350499 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:47.350610 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:47.351397 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:47.351401 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:47.351731 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:40:47.351864 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:47.351872 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.353543 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.353546 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:47.354406 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:47.354418 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.355389 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.355393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:47.355577 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:47.356056 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:40:47.356073 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:40:47.358045 (+ 1972us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:40:47.361395 (+ 3350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:47.361399 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:47.361403 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:47.361862 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:47.362230 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:47.362234 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:47.362237 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:47.362408 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:47.362955 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:40:47.362959 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:40:47.363104 (+ 145us) client_negotiation.cc:769] Sending connection context
0808 11:40:47.363259 (+ 155us) client_negotiation.cc:240] Negotiation successful
0808 11:40:47.363531 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":4}
I20260808 11:40:47.363921 13721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.350374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39564 (local address 127.24.76.125:46461)
0808 11:40:47.350943 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:47.350951 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:47.350982 (+ 31us) server_negotiation.cc:407] Connection header received
0808 11:40:47.351033 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:47.351040 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:47.351244 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:47.351378 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:47.352008 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.353409 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.354535 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.355264 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.355382 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:47.358244 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:47.358256 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:47.358271 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:47.358313 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:47.361270 (+ 2957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:47.362006 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:47.362011 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:47.362014 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:47.362123 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:47.362508 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:47.362512 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:47.362513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:47.362837 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:47.363005 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:47.363379 (+ 374us) server_negotiation.cc:299] Negotiation successful
0808 11:40:47.363628 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":115,"server-negotiator.queue_time_us":400,"thread_start_us":259,"threads_started":1}
I20260808 11:40:47.365991 13712 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.76.124" port: 43521 } has no permanent_uuid. Determining permanent_uuid...
I20260808 11:40:47.366988 13715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.366531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:43521 (local address 127.0.0.1:53166)
0808 11:40:47.366722 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:47.366832 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:43521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 11:40:47.367229 13643 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.76.124:43521: connect: Connection refused (error 111)
I20260808 11:40:47.368726 13715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.368378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:43521 (local address 127.0.0.1:53170)
0808 11:40:47.368531 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:47.368617 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:43521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 11:40:47.368950 13712 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.124:43521: Network error: Client connection negotiation failed: client connection to 127.24.76.124:43521: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.124.keytab.
Entry for principal kudu/127.24.76.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.124.keytab.
I20260808 11:40:47.381125 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.124:43521
--webserver_interface=127.24.76.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.124.keytab
--principal=kudu/127.24.76.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:39067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:43431,127.24.76.125:46461,127.24.76.124:43521
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:40:47.418227 13712 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.124" port: 43521 } attempt: 1
I20260808 11:40:47.419438 13715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.418876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:43521 (local address 127.0.0.1:53180)
0808 11:40:47.419123 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:47.419252 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:43521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 11:40:47.420521 13715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.420108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:43521 (local address 127.0.0.1:53192)
0808 11:40:47.420272 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:47.420374 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:43521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 11:40:47.420763 13712 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.124:43521: Network error: Client connection negotiation failed: client connection to 127.24.76.124:43521: connect: Connection refused (error 111)
I20260808 11:40:47.488978 13712 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.124" port: 43521 } attempt: 2
I20260808 11:40:47.490279 13715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.489621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:43521 (local address 127.0.0.1:53208)
0808 11:40:47.489937 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:47.490063 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:43521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260808 11:40:47.491427 13715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.491011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:43521 (local address 127.0.0.1:53216)
0808 11:40:47.491168 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:47.491262 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:43521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 11:40:47.491668 13712 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.124:43521: Network error: Client connection negotiation failed: client connection to 127.24.76.124:43521: connect: Connection refused (error 111)
I20260808 11:40:47.507023 13609 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.125" port: 46461 } attempt: 5
Aug 08 11:40:47 dist-test-slave-tc2s krb5kdc[13483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189246, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for kudu/127.24.76.125@KRBTEST.COM
I20260808 11:40:47.520733 13721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.507702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39568 (local address 127.24.76.125:46461)
0808 11:40:47.507861 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:47.507866 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:47.507884 (+ 18us) server_negotiation.cc:407] Connection header received
0808 11:40:47.508078 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:47.508082 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:47.508141 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:47.508219 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:47.509177 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.509841 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.511534 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.511964 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.512054 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:47.515704 (+ 3650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:47.515709 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:47.515712 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:47.515744 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:47.518176 (+ 2432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:47.518895 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:47.518899 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:47.518901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:47.519027 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:47.519415 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:47.519419 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:47.519421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:47.519692 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:47.519801 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:47.520250 (+ 449us) server_negotiation.cc:299] Negotiation successful
0808 11:40:47.520480 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 11:40:47.520790 13612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.507592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:46461 (local address 127.0.0.1:39568)
0808 11:40:47.507797 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:47.507827 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:47.507963 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:47.508280 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:47.508285 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:47.508781 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:40:47.509041 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:47.509054 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.509987 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.509992 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:47.511396 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:47.511413 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.512113 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.512119 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:47.512313 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:47.512940 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:40:47.512963 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:40:47.515516 (+ 2553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:40:47.518321 (+ 2805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:47.518326 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:47.518330 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:47.518789 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:47.519134 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:47.519138 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:47.519142 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:47.519309 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:47.519819 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:40:47.519824 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:40:47.519991 (+ 167us) client_negotiation.cc:769] Sending connection context
0808 11:40:47.520171 (+ 180us) client_negotiation.cc:240] Negotiation successful
0808 11:40:47.520461 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 11:40:47.521982 13609 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.76.124" port: 43521 } has no permanent_uuid. Determining permanent_uuid...
I20260808 11:40:47.522974 13612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.522531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:43521 (local address 127.0.0.1:53226)
0808 11:40:47.522703 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:47.522798 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:43521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 11:40:47.523943 13612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.523544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:43521 (local address 127.0.0.1:53232)
0808 11:40:47.523703 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:47.523814 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:43521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 11:40:47.524194 13609 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.124:43521: Network error: Client connection negotiation failed: client connection to 127.24.76.124:43521: connect: Connection refused (error 111)
I20260808 11:40:47.575410 13609 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.124" port: 43521 } attempt: 1
I20260808 11:40:47.576613 13612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.576129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:43521 (local address 127.0.0.1:53234)
0808 11:40:47.576335 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:47.576446 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:43521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260808 11:40:47.577831 13612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.577412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:43521 (local address 127.0.0.1:53238)
0808 11:40:47.577623 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:47.577713 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:43521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260808 11:40:47.578083 13609 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.124:43521: Network error: Client connection negotiation failed: client connection to 127.24.76.124:43521: connect: Connection refused (error 111)
I20260808 11:40:47.582885 13712 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.124" port: 43521 } attempt: 3
I20260808 11:40:47.583946 13715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.583432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:43521 (local address 127.0.0.1:53240)
0808 11:40:47.583642 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:47.583748 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:43521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 11:40:47.585112 13715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.584628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:43521 (local address 127.0.0.1:53246)
0808 11:40:47.584867 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:47.584974 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:43521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
W20260808 11:40:47.585350 13712 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.124:43521: Network error: Client connection negotiation failed: client connection to 127.24.76.124:43521: connect: Connection refused (error 111)
I20260808 11:40:47.646293 13609 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.124" port: 43521 } attempt: 2
I20260808 11:40:47.647442 13612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.646949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:43521 (local address 127.0.0.1:53262)
0808 11:40:47.647179 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:47.647268 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:43521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 11:40:47.648509 13612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.648156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:43521 (local address 127.0.0.1:53274)
0808 11:40:47.648275 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:47.648396 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:43521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 11:40:47.648729 13609 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.124:43521: Network error: Client connection negotiation failed: client connection to 127.24.76.124:43521: connect: Connection refused (error 111)
W20260808 11:40:47.713863 13725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:40:47.714140 13725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:40:47.714236 13725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:40:47.721206 13725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:40:47.721298 13725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:40:47.721357 13725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:40:47.721412 13725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:40:47.721455 13725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:40:47.728574 13712 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.124" port: 43521 } attempt: 4
I20260808 11:40:47.729773 13715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.729228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:43521 (local address 127.0.0.1:53286)
0808 11:40:47.729471 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:47.729591 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:43521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 11:40:47.730747 13715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.730413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:43521 (local address 127.0.0.1:53302)
0808 11:40:47.730543 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:47.730622 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:43521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 11:40:47.730947 13712 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.124:43521: Network error: Client connection negotiation failed: client connection to 127.24.76.124:43521: connect: Connection refused (error 111)
I20260808 11:40:47.731454 13725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:39067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:43431,127.24.76.125:46461,127.24.76.124:43521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.124.keytab
--principal=kudu/127.24.76.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.124:43521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.76.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:40:47.732605 13725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:40:47.733769 13725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:40:47.745106 13731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:47.745769 13733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:40:47.750104 13725 server_base.cc:1034] running on GCE node
W20260808 11:40:47.750799 13735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:40:47.751575 13725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:40:47.752893 13725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:40:47.754101 13725 hybrid_clock.cc:648] HybridClock initialized: now 1786189247754047 us; error 40 us; skew 500 ppm
I20260808 11:40:47.754415 13725 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:40:47.754904 13609 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.124" port: 43521 } attempt: 3
I20260808 11:40:47.755873 13612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.755462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:43521 (local address 127.0.0.1:53314)
0808 11:40:47.755639 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:47.755721 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:43521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 11:40:47.756809 13612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.756451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:43521 (local address 127.0.0.1:53324)
0808 11:40:47.756616 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:47.756687 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:43521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 11:40:47.757028 13609 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.124:43521: Network error: Client connection negotiation failed: client connection to 127.24.76.124:43521: connect: Connection refused (error 111)
Aug 08 11:40:47 dist-test-slave-tc2s krb5kdc[13483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189247, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:40:47.757627 13725 init.cc:377] Logged in from keytab as kudu/127.24.76.124@KRBTEST.COM (short username kudu)
I20260808 11:40:47.759110 13725 webserver.cc:460] Webserver started at http://127.24.76.124:43011/ using document root <none> and password file <none>
I20260808 11:40:47.759766 13725 fs_manager.cc:359] Metadata directory not provided
I20260808 11:40:47.759871 13725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:40:47.760201 13725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:40:47.763151 13725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-2/data/instance:
uuid: "6724e2a931ab42b6bd6930d2eb87bbd5"
format_stamp: "Formatted at 2026-08-08 11:40:47 on dist-test-slave-tc2s"
server_key: "cdb1c6f2168cf238dc030c3c795bcfeb"
server_key_iv: "7e465231c46d0dc63b8e14dbc5726953"
server_key_version: "encryptionkey@0"
I20260808 11:40:47.763921 13725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-2/wal/instance:
uuid: "6724e2a931ab42b6bd6930d2eb87bbd5"
format_stamp: "Formatted at 2026-08-08 11:40:47 on dist-test-slave-tc2s"
server_key: "cdb1c6f2168cf238dc030c3c795bcfeb"
server_key_iv: "7e465231c46d0dc63b8e14dbc5726953"
server_key_version: "encryptionkey@0"
I20260808 11:40:47.768656 13725 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260808 11:40:47.771929 13747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:47.773113 13725 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 11:40:47.773265 13725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-2/wal
uuid: "6724e2a931ab42b6bd6930d2eb87bbd5"
format_stamp: "Formatted at 2026-08-08 11:40:47 on dist-test-slave-tc2s"
server_key: "cdb1c6f2168cf238dc030c3c795bcfeb"
server_key_iv: "7e465231c46d0dc63b8e14dbc5726953"
server_key_version: "encryptionkey@0"
I20260808 11:40:47.773435 13725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:40:47.810541 13725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:40:47.814440 13725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:40:47.814706 13725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:40:47.831280 13725 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.124:43521
I20260808 11:40:47.831321 13826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.124:43521 every 8 connection(s)
I20260808 11:40:47.832557 13725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-2/data/info.pb
I20260808 11:40:47.834971 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 13725
I20260808 11:40:47.835147 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786189042529715-24881-0/minicluster-data/master-2/wal/instance
I20260808 11:40:47.835657 24881 external_mini_cluster.cc:1428] Setting key e79becd83ca6d812f62926165371e5c1
I20260808 11:40:47.836288 13828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:40:47.844689 13828 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.76.126" port: 43431 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:40:47 dist-test-slave-tc2s krb5kdc[13483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189246, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.124@KRBTEST.COM
Aug 08 11:40:47 dist-test-slave-tc2s krb5kdc[13483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189247, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.124@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:40:47.857087 13832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.837515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53340 (local address 127.24.76.124:43521)
0808 11:40:47.838310 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:47.838321 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:47.838361 (+ 40us) server_negotiation.cc:407] Connection header received
0808 11:40:47.839879 (+ 1518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:47.839900 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:47.840264 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:47.840652 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:47.842214 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.843551 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.844881 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.845581 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.845737 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:47.850455 (+ 4718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:47.850462 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:47.850471 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:47.850510 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:47.853356 (+ 2846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:47.854019 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:47.854025 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:47.854031 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:47.854189 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:47.854556 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:47.854559 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:47.854562 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:47.855092 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:47.855369 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:47.855803 (+ 434us) server_negotiation.cc:299] Negotiation successful
0808 11:40:47.856181 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":367,"threads_started":1}
I20260808 11:40:47.861685 13613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.847056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32852 (local address 127.24.76.126:43431)
0808 11:40:47.847256 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:47.847263 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:47.847972 (+ 709us) server_negotiation.cc:407] Connection header received
0808 11:40:47.848144 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:47.848148 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:47.848211 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:47.848290 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:47.849486 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.850522 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.851898 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.852326 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.852433 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:47.855393 (+ 2960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:47.855397 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:47.855401 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:47.855434 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:47.857971 (+ 2537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:47.859606 (+ 1635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:47.859611 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:47.859615 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:47.859800 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:47.860219 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:47.860224 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:47.860228 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:47.860559 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:47.860713 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:47.861147 (+ 434us) server_negotiation.cc:299] Negotiation successful
0808 11:40:47.861343 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 11:40:47.861683 13833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.847024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:43431 (local address 127.0.0.1:32852)
0808 11:40:47.847822 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:47.847869 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:47.848034 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:47.848378 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:47.848384 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:47.849052 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:40:47.849334 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:47.849345 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.850694 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.850698 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:47.851758 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:47.851771 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.852519 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.852529 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:47.852679 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:47.853257 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:40:47.853273 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:40:47.855219 (+ 1946us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:40:47.858899 (+ 3680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:47.858909 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:47.858916 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:47.859495 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:47.859941 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:47.859954 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:47.859957 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:47.860112 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:47.860694 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:40:47.860701 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:40:47.860859 (+ 158us) client_negotiation.cc:769] Sending connection context
0808 11:40:47.861016 (+ 157us) client_negotiation.cc:240] Negotiation successful
0808 11:40:47.861295 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":366,"threads_started":1}
I20260808 11:40:47.863096 13828 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.76.125" port: 46461 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:40:47 dist-test-slave-tc2s krb5kdc[13483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189246, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
Aug 08 11:40:47 dist-test-slave-tc2s krb5kdc[13483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189247, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.124@KRBTEST.COM for kudu/127.24.76.125@KRBTEST.COM
I20260808 11:40:47.873821 13835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.859746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32866 (local address 127.24.76.126:43431)
0808 11:40:47.860623 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:47.860627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:47.860645 (+ 18us) server_negotiation.cc:407] Connection header received
0808 11:40:47.860682 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:47.860686 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:47.860732 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:47.860815 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:47.862086 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.862894 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.864300 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.864697 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.864797 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:47.868593 (+ 3796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:47.868598 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:47.868601 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:47.868636 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:47.871327 (+ 2691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:47.871979 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:47.871984 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:47.871988 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:47.872100 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:47.872440 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:47.872445 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:47.872449 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:47.872781 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:47.873041 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:47.873246 (+ 205us) server_negotiation.cc:299] Negotiation successful
0808 11:40:47.873430 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":757,"thread_start_us":577,"threads_started":1}
I20260808 11:40:47.875017 13833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.863696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:46461 (local address 127.0.0.1:39570)
0808 11:40:47.863852 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:47.863870 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:47.863965 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:47.864251 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:47.864256 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:47.864587 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:40:47.864703 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:47.864710 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.865643 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.865647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:47.866486 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:47.866497 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.867165 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.867171 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:47.867343 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:47.867821 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:40:47.867836 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:40:47.869832 (+ 1996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:40:47.872588 (+ 2756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:47.872593 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:47.872596 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:47.873063 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:47.873474 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:47.873501 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:47.873505 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:47.873673 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:47.874240 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:40:47.874245 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:40:47.874386 (+ 141us) client_negotiation.cc:769] Sending connection context
0808 11:40:47.874525 (+ 139us) client_negotiation.cc:240] Negotiation successful
0808 11:40:47.874725 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 11:40:47.875161 13721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.863773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39570 (local address 127.24.76.125:46461)
0808 11:40:47.863940 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:47.863946 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:47.863964 (+ 18us) server_negotiation.cc:407] Connection header received
0808 11:40:47.864074 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:47.864080 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:47.864143 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:47.864222 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:47.864846 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.865506 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.866615 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.867030 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.867134 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:47.870027 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:47.870031 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:47.870034 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:47.870065 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:47.872476 (+ 2411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:47.873173 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:47.873178 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:47.873181 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:47.873345 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:47.873772 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:47.873776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:47.873778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:47.874129 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:47.874245 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:47.874643 (+ 398us) server_negotiation.cc:299] Negotiation successful
0808 11:40:47.874863 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 11:40:47.875946 13828 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.76.124" port: 43521 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:40:47 dist-test-slave-tc2s krb5kdc[13483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189247, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.124@KRBTEST.COM for kudu/127.24.76.124@KRBTEST.COM
Aug 08 11:40:47 dist-test-slave-tc2s krb5kdc[13483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189246, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.125@KRBTEST.COM
I20260808 11:40:47.888015 13833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.876530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:43521 (local address 127.0.0.1:53348)
0808 11:40:47.876683 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:47.876700 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:47.876783 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:47.877078 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:47.877080 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:47.877374 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:40:47.877518 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:47.877528 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.878488 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.878493 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:47.879284 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:47.879295 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.880063 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.880069 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:47.880221 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:47.880749 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:40:47.880770 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:40:47.883182 (+ 2412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:40:47.885872 (+ 2690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:47.885875 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:47.885878 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:47.886235 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:47.886565 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:47.886570 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:47.886574 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:47.886686 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:47.887142 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:40:47.887146 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:40:47.887266 (+ 120us) client_negotiation.cc:769] Sending connection context
0808 11:40:47.887392 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 11:40:47.887648 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 11:40:47.888155 13832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.876683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53348 (local address 127.24.76.124:43521)
0808 11:40:47.876833 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:47.876838 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:47.876856 (+ 18us) server_negotiation.cc:407] Connection header received
0808 11:40:47.876900 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:47.876905 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:47.876969 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:47.877087 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:47.877652 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.878363 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.879419 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.879915 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.880009 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:47.883342 (+ 3333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:47.883346 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:47.883349 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:47.883375 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:47.885769 (+ 2394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:47.886339 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:47.886344 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:47.886348 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:47.886462 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:47.886780 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:47.886785 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:47.886788 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:47.887037 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:47.887157 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:47.887501 (+ 344us) server_negotiation.cc:299] Negotiation successful
0808 11:40:47.887814 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":96,"server-negotiator.queue_time_us":48}
I20260808 11:40:47.889568 13721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.875687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39578 (local address 127.24.76.125:46461)
0808 11:40:47.875816 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:47.875820 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:47.875833 (+ 13us) server_negotiation.cc:407] Connection header received
0808 11:40:47.875874 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:47.875878 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:47.875940 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:47.876040 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:47.877538 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.878379 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.879668 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.880134 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.880246 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:47.884667 (+ 4421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:47.884671 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:47.884674 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:47.884702 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:47.887033 (+ 2331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:47.887620 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:47.887624 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:47.887626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:47.887748 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:47.888273 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:47.888279 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:47.888282 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:47.888538 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:47.888779 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:47.889082 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 11:40:47.889289 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 11:40:47.893332 13828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6724e2a931ab42b6bd6930d2eb87bbd5: Bootstrap starting.
I20260808 11:40:47.894006 24881 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:40:47.897142 13828 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6724e2a931ab42b6bd6930d2eb87bbd5: Neither blocks nor log segments found. Creating new log.
I20260808 11:40:47.898499 13828 log.cc:824] T 00000000000000000000000000000000 P 6724e2a931ab42b6bd6930d2eb87bbd5: Log is configured to *not* fsync() on all Append() calls
I20260808 11:40:47.901561 13828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6724e2a931ab42b6bd6930d2eb87bbd5: No bootstrap required, opened a new log
I20260808 11:40:47.912672 13828 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6724e2a931ab42b6bd6930d2eb87bbd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76572ad082cd49758fe05a17dfb0d686" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 43431 } } peers { permanent_uuid: "5f62a83d9dc34126b38fca7d1f0413c3" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 46461 } } peers { permanent_uuid: "6724e2a931ab42b6bd6930d2eb87bbd5" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 43521 } }
I20260808 11:40:47.913090 13828 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6724e2a931ab42b6bd6930d2eb87bbd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:40:47.913193 13828 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6724e2a931ab42b6bd6930d2eb87bbd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6724e2a931ab42b6bd6930d2eb87bbd5, State: Initialized, Role: FOLLOWER
I20260808 11:40:47.914472 13828 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6724e2a931ab42b6bd6930d2eb87bbd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76572ad082cd49758fe05a17dfb0d686" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 43431 } } peers { permanent_uuid: "5f62a83d9dc34126b38fca7d1f0413c3" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 46461 } } peers { permanent_uuid: "6724e2a931ab42b6bd6930d2eb87bbd5" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 43521 } }
I20260808 11:40:47.917096 13835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.900139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32870 (local address 127.24.76.126:43431)
0808 11:40:47.900315 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:47.900321 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:47.900374 (+ 53us) server_negotiation.cc:407] Connection header received
0808 11:40:47.900612 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:47.900617 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:47.900684 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:47.900764 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:47.902681 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.903683 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.906097 (+ 2414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.906741 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.906834 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:47.910566 (+ 3732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:47.910572 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:47.910576 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:47.910614 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:47.913864 (+ 3250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:47.914696 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:47.914701 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:47.914704 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:47.914854 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:47.915369 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:47.915375 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:47.915379 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:47.915714 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:47.915937 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:47.916545 (+ 608us) server_negotiation.cc:299] Negotiation successful
0808 11:40:47.916756 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 11:40:47.920150 13721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.899720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39594 (local address 127.24.76.125:46461)
0808 11:40:47.899874 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:47.899879 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:47.902490 (+ 2611us) server_negotiation.cc:407] Connection header received
0808 11:40:47.902732 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:47.902737 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:47.902806 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:47.902867 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:47.905126 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.906148 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.908023 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.908640 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.908736 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:47.912538 (+ 3802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:47.912545 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:47.912550 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:47.912587 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:47.916275 (+ 3688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:47.917477 (+ 1202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:47.917495 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:47.917499 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:47.917822 (+ 323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:47.918295 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:47.918300 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:47.918303 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:47.918633 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:47.918865 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:47.919517 (+ 652us) server_negotiation.cc:299] Negotiation successful
0808 11:40:47.919732 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 11:40:47.920687 13848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6724e2a931ab42b6bd6930d2eb87bbd5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76572ad082cd49758fe05a17dfb0d686" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 43431 } } peers { permanent_uuid: "5f62a83d9dc34126b38fca7d1f0413c3" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 46461 } } peers { permanent_uuid: "6724e2a931ab42b6bd6930d2eb87bbd5" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 43521 } } }
I20260808 11:40:47.921023 13848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6724e2a931ab42b6bd6930d2eb87bbd5 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:40:47.921799 13828 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6724e2a931ab42b6bd6930d2eb87bbd5 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 11:40:47.931159 13865 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6724e2a931ab42b6bd6930d2eb87bbd5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 11:40:47.931300 13865 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 11:40:47.932370 13832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.903886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53350 (local address 127.24.76.124:43521)
0808 11:40:47.904085 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:47.904091 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:47.914033 (+ 9942us) server_negotiation.cc:407] Connection header received
0808 11:40:47.914213 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:47.914218 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:47.914288 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:47.914352 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:47.916221 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.918439 (+ 2218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.921469 (+ 3030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.922179 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.922306 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:47.925394 (+ 3088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:47.925403 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:47.925408 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:47.925452 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:47.929042 (+ 3590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:47.929913 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:47.929918 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:47.929922 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:47.930127 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:47.930608 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:47.930617 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:47.930620 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:47.930977 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:47.931222 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:47.931780 (+ 558us) server_negotiation.cc:299] Negotiation successful
0808 11:40:47.932000 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 11:40:47.934237 13609 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.124" port: 43521 } attempt: 4
I20260808 11:40:47.928512 13725 master_runner.cc:186] Error getting master registration for 127.24.76.126:43431: OK, instance_id { permanent_uuid: "76572ad082cd49758fe05a17dfb0d686" instance_seqno: 1786189246800527 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 11:40:47.938395 13725 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 08 11:40:47 dist-test-slave-tc2s krb5kdc[13483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189246, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for kudu/127.24.76.124@KRBTEST.COM
I20260808 11:40:47.949240 13832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.934810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53366 (local address 127.24.76.124:43521)
0808 11:40:47.934963 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:47.934968 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:47.935086 (+ 118us) server_negotiation.cc:407] Connection header received
0808 11:40:47.935230 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:47.935234 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:47.935303 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:47.935413 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:47.936221 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.937390 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.939086 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.939732 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.939919 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:47.943258 (+ 3339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:47.943264 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:47.943268 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:47.943301 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:47.945865 (+ 2564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:47.947570 (+ 1705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:47.947575 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:47.947577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:47.947719 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:47.948098 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:47.948101 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:47.948103 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:47.948331 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:47.948438 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:47.948851 (+ 413us) server_negotiation.cc:299] Negotiation successful
0808 11:40:47.948999 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 11:40:47.949414 13612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:47.934765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:43521 (local address 127.0.0.1:53366)
0808 11:40:47.935001 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:47.935018 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:47.935128 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:47.935437 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:47.935442 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:47.935893 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:40:47.936081 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:47.936094 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.937568 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.937574 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:47.938928 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:47.938944 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:47.939927 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:47.939933 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:47.940238 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:47.940752 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:40:47.940769 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:40:47.943078 (+ 2309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:40:47.946003 (+ 2925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:47.946009 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:47.946013 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:47.947429 (+ 1416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:47.947829 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:47.947834 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:47.947838 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:47.947999 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:47.948439 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:40:47.948452 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:40:47.948634 (+ 182us) client_negotiation.cc:769] Sending connection context
0808 11:40:47.948788 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 11:40:47.949050 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134}
I20260808 11:40:47.953523 13609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76572ad082cd49758fe05a17dfb0d686: Bootstrap starting.
I20260808 11:40:47.957080 13609 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76572ad082cd49758fe05a17dfb0d686: Neither blocks nor log segments found. Creating new log.
I20260808 11:40:47.958091 13609 log.cc:824] T 00000000000000000000000000000000 P 76572ad082cd49758fe05a17dfb0d686: Log is configured to *not* fsync() on all Append() calls
I20260808 11:40:47.960724 13609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76572ad082cd49758fe05a17dfb0d686: No bootstrap required, opened a new log
I20260808 11:40:47.967967 13609 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76572ad082cd49758fe05a17dfb0d686 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76572ad082cd49758fe05a17dfb0d686" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 43431 } } peers { permanent_uuid: "5f62a83d9dc34126b38fca7d1f0413c3" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 46461 } } peers { permanent_uuid: "6724e2a931ab42b6bd6930d2eb87bbd5" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 43521 } }
I20260808 11:40:47.968250 13609 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76572ad082cd49758fe05a17dfb0d686 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:40:47.968357 13609 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76572ad082cd49758fe05a17dfb0d686 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76572ad082cd49758fe05a17dfb0d686, State: Initialized, Role: FOLLOWER
I20260808 11:40:47.968948 13609 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76572ad082cd49758fe05a17dfb0d686 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76572ad082cd49758fe05a17dfb0d686" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 43431 } } peers { permanent_uuid: "5f62a83d9dc34126b38fca7d1f0413c3" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 46461 } } peers { permanent_uuid: "6724e2a931ab42b6bd6930d2eb87bbd5" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 43521 } }
I20260808 11:40:47.970119 13872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76572ad082cd49758fe05a17dfb0d686 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76572ad082cd49758fe05a17dfb0d686" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 43431 } } peers { permanent_uuid: "5f62a83d9dc34126b38fca7d1f0413c3" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 46461 } } peers { permanent_uuid: "6724e2a931ab42b6bd6930d2eb87bbd5" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 43521 } } }
I20260808 11:40:47.970575 13872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76572ad082cd49758fe05a17dfb0d686 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:40:47.971272 13609 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76572ad082cd49758fe05a17dfb0d686 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 11:40:47.992729 13891 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 76572ad082cd49758fe05a17dfb0d686: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 11:40:47.992879 13891 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 11:40:47.986620 13501 master_runner.cc:186] Error getting master registration for 127.24.76.125:46461: OK, instance_id { permanent_uuid: "5f62a83d9dc34126b38fca7d1f0413c3" instance_seqno: 1786189247308435 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 11:40:47.997469 13501 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 11:40:48.030138 13712 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.124" port: 43521 } attempt: 5
Aug 08 11:40:48 dist-test-slave-tc2s krb5kdc[13483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189247, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.125@KRBTEST.COM for kudu/127.24.76.124@KRBTEST.COM
W20260808 11:40:48.040829 13536 tablet.cc:2369] T 00000000000000000000000000000000 P 76572ad082cd49758fe05a17dfb0d686: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:40:48.044338 13715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:48.030665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:43521 (local address 127.0.0.1:53378)
0808 11:40:48.030917 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:48.030936 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:48.031055 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:48.031355 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:48.031360 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:48.031771 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:40:48.031933 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:48.031942 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:48.033539 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:48.033544 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:48.034895 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:48.034912 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:48.035756 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:48.035761 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:48.035964 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:48.036515 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:40:48.036533 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:40:48.038541 (+ 2008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:40:48.041990 (+ 3449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:48.041996 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:48.042000 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:48.042372 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:48.042738 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:48.042743 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:48.042747 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:48.042877 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:48.043378 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:40:48.043382 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:40:48.043521 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 11:40:48.043653 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 11:40:48.043937 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131}
I20260808 11:40:48.044349 13832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:48.030754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53378 (local address 127.24.76.124:43521)
0808 11:40:48.030916 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:48.030921 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:48.031032 (+ 111us) server_negotiation.cc:407] Connection header received
0808 11:40:48.031160 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:48.031166 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:48.031226 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:48.031303 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:48.032075 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:48.033156 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:48.035068 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:48.035616 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:48.035731 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:48.038702 (+ 2971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:48.038707 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:48.038710 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:48.038739 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:48.041851 (+ 3112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:48.042493 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:48.042498 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:48.042500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:48.042615 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:48.042979 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:48.042984 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:48.042987 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:48.043259 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:48.043413 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:48.043768 (+ 355us) server_negotiation.cc:299] Negotiation successful
0808 11:40:48.043961 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 11:40:48.049214 13712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f62a83d9dc34126b38fca7d1f0413c3: Bootstrap starting.
I20260808 11:40:48.052944 13712 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f62a83d9dc34126b38fca7d1f0413c3: Neither blocks nor log segments found. Creating new log.
I20260808 11:40:48.053972 13712 log.cc:824] T 00000000000000000000000000000000 P 5f62a83d9dc34126b38fca7d1f0413c3: Log is configured to *not* fsync() on all Append() calls
I20260808 11:40:48.056397 13712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f62a83d9dc34126b38fca7d1f0413c3: No bootstrap required, opened a new log
I20260808 11:40:48.063594 13712 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f62a83d9dc34126b38fca7d1f0413c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76572ad082cd49758fe05a17dfb0d686" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 43431 } } peers { permanent_uuid: "5f62a83d9dc34126b38fca7d1f0413c3" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 46461 } } peers { permanent_uuid: "6724e2a931ab42b6bd6930d2eb87bbd5" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 43521 } }
I20260808 11:40:48.063886 13712 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f62a83d9dc34126b38fca7d1f0413c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:40:48.064006 13712 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f62a83d9dc34126b38fca7d1f0413c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f62a83d9dc34126b38fca7d1f0413c3, State: Initialized, Role: FOLLOWER
I20260808 11:40:48.064551 13712 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f62a83d9dc34126b38fca7d1f0413c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76572ad082cd49758fe05a17dfb0d686" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 43431 } } peers { permanent_uuid: "5f62a83d9dc34126b38fca7d1f0413c3" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 46461 } } peers { permanent_uuid: "6724e2a931ab42b6bd6930d2eb87bbd5" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 43521 } }
W20260808 11:40:48.066632 13755 tablet.cc:2369] T 00000000000000000000000000000000 P 6724e2a931ab42b6bd6930d2eb87bbd5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:40:48.069121 13898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f62a83d9dc34126b38fca7d1f0413c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76572ad082cd49758fe05a17dfb0d686" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 43431 } } peers { permanent_uuid: "5f62a83d9dc34126b38fca7d1f0413c3" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 46461 } } peers { permanent_uuid: "6724e2a931ab42b6bd6930d2eb87bbd5" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 43521 } } }
I20260808 11:40:48.069661 13898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f62a83d9dc34126b38fca7d1f0413c3 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:40:48.070349 13712 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f62a83d9dc34126b38fca7d1f0413c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:40:48.090750 13898 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f62a83d9dc34126b38fca7d1f0413c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:40:48.090939 13619 master_runner.cc:263] No leader master found from master 5f62a83d9dc34126b38fca7d1f0413c3
I20260808 11:40:48.091099 13619 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 11:40:48.091090 13898 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f62a83d9dc34126b38fca7d1f0413c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76572ad082cd49758fe05a17dfb0d686" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 43431 } } peers { permanent_uuid: "5f62a83d9dc34126b38fca7d1f0413c3" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 46461 } } peers { permanent_uuid: "6724e2a931ab42b6bd6930d2eb87bbd5" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 43521 } }
I20260808 11:40:48.092247 13898 leader_election.cc:288] T 00000000000000000000000000000000 P 5f62a83d9dc34126b38fca7d1f0413c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76572ad082cd49758fe05a17dfb0d686 (127.24.76.126:43431), 6724e2a931ab42b6bd6930d2eb87bbd5 (127.24.76.124:43521)
I20260808 11:40:48.092857 13572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5f62a83d9dc34126b38fca7d1f0413c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76572ad082cd49758fe05a17dfb0d686" is_pre_election: true
I20260808 11:40:48.093182 13793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5f62a83d9dc34126b38fca7d1f0413c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6724e2a931ab42b6bd6930d2eb87bbd5" is_pre_election: true
I20260808 11:40:48.093503 13572 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 76572ad082cd49758fe05a17dfb0d686 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f62a83d9dc34126b38fca7d1f0413c3 in term 0.
I20260808 11:40:48.093740 13793 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6724e2a931ab42b6bd6930d2eb87bbd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f62a83d9dc34126b38fca7d1f0413c3 in term 0.
I20260808 11:40:48.094115 13642 leader_election.cc:302] T 00000000000000000000000000000000 P 5f62a83d9dc34126b38fca7d1f0413c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f62a83d9dc34126b38fca7d1f0413c3, 76572ad082cd49758fe05a17dfb0d686; no voters:
I20260808 11:40:48.094492 13898 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f62a83d9dc34126b38fca7d1f0413c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:40:48.094633 13898 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f62a83d9dc34126b38fca7d1f0413c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:40:48.094728 13898 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f62a83d9dc34126b38fca7d1f0413c3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:40:48.098316 13898 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f62a83d9dc34126b38fca7d1f0413c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76572ad082cd49758fe05a17dfb0d686" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 43431 } } peers { permanent_uuid: "5f62a83d9dc34126b38fca7d1f0413c3" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 46461 } } peers { permanent_uuid: "6724e2a931ab42b6bd6930d2eb87bbd5" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 43521 } }
I20260808 11:40:48.099339 13572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5f62a83d9dc34126b38fca7d1f0413c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76572ad082cd49758fe05a17dfb0d686"
I20260808 11:40:48.099551 13572 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76572ad082cd49758fe05a17dfb0d686 [term 0 FOLLOWER]: Advancing to term 1
W20260808 11:40:48.101809 13910 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5f62a83d9dc34126b38fca7d1f0413c3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 11:40:48.101984 13910 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 11:40:48.102806 13572 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 76572ad082cd49758fe05a17dfb0d686 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f62a83d9dc34126b38fca7d1f0413c3 in term 1.
I20260808 11:40:48.103233 13642 leader_election.cc:302] T 00000000000000000000000000000000 P 5f62a83d9dc34126b38fca7d1f0413c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f62a83d9dc34126b38fca7d1f0413c3, 76572ad082cd49758fe05a17dfb0d686; no voters:
I20260808 11:40:48.105574 13898 leader_election.cc:288] T 00000000000000000000000000000000 P 5f62a83d9dc34126b38fca7d1f0413c3 [CANDIDATE]: Term 1 election: Requested vote from peers 76572ad082cd49758fe05a17dfb0d686 (127.24.76.126:43431), 6724e2a931ab42b6bd6930d2eb87bbd5 (127.24.76.124:43521)
I20260808 11:40:48.105746 13898 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f62a83d9dc34126b38fca7d1f0413c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:40:48.105873 13898 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f62a83d9dc34126b38fca7d1f0413c3 [term 1 LEADER]: Becoming Leader. State: Replica: 5f62a83d9dc34126b38fca7d1f0413c3, State: Running, Role: LEADER
I20260808 11:40:48.106401 13898 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f62a83d9dc34126b38fca7d1f0413c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76572ad082cd49758fe05a17dfb0d686" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 43431 } } peers { permanent_uuid: "5f62a83d9dc34126b38fca7d1f0413c3" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 46461 } } peers { permanent_uuid: "6724e2a931ab42b6bd6930d2eb87bbd5" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 43521 } }
I20260808 11:40:48.106665 13793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5f62a83d9dc34126b38fca7d1f0413c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6724e2a931ab42b6bd6930d2eb87bbd5"
I20260808 11:40:48.106856 13793 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6724e2a931ab42b6bd6930d2eb87bbd5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:40:48.110129 13793 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6724e2a931ab42b6bd6930d2eb87bbd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f62a83d9dc34126b38fca7d1f0413c3 in term 1.
I20260808 11:40:48.117141 13912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f62a83d9dc34126b38fca7d1f0413c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f62a83d9dc34126b38fca7d1f0413c3. Latest consensus state: current_term: 1 leader_uuid: "5f62a83d9dc34126b38fca7d1f0413c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76572ad082cd49758fe05a17dfb0d686" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 43431 } } peers { permanent_uuid: "5f62a83d9dc34126b38fca7d1f0413c3" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 46461 } } peers { permanent_uuid: "6724e2a931ab42b6bd6930d2eb87bbd5" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 43521 } } }
I20260808 11:40:48.117547 13912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f62a83d9dc34126b38fca7d1f0413c3 [sys.catalog]: This master's current role is: LEADER
I20260808 11:40:48.118405 13915 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:40:48.120625 13915 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:40:48.135877 13793 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6724e2a931ab42b6bd6930d2eb87bbd5 [term 1 FOLLOWER]: Refusing update from remote peer 5f62a83d9dc34126b38fca7d1f0413c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:40:48.135901 13572 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 76572ad082cd49758fe05a17dfb0d686 [term 1 FOLLOWER]: Refusing update from remote peer 5f62a83d9dc34126b38fca7d1f0413c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:40:48.136804 13912 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5f62a83d9dc34126b38fca7d1f0413c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6724e2a931ab42b6bd6930d2eb87bbd5" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 43521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:40:48.141314 13912 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5f62a83d9dc34126b38fca7d1f0413c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76572ad082cd49758fe05a17dfb0d686" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 43431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:40:48.151234 13848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6724e2a931ab42b6bd6930d2eb87bbd5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f62a83d9dc34126b38fca7d1f0413c3. Latest consensus state: current_term: 1 leader_uuid: "5f62a83d9dc34126b38fca7d1f0413c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76572ad082cd49758fe05a17dfb0d686" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 43431 } } peers { permanent_uuid: "5f62a83d9dc34126b38fca7d1f0413c3" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 46461 } } peers { permanent_uuid: "6724e2a931ab42b6bd6930d2eb87bbd5" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 43521 } } }
I20260808 11:40:48.151551 13848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6724e2a931ab42b6bd6930d2eb87bbd5 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:40:48.157531 13918 mvcc.cc:205] Tried to move back new op lower bound from 7316231160351866880 to 7316231160255668224. Current Snapshot: MvccSnapshot[applied={T|T < 7316231160351866880}]
I20260808 11:40:48.162320 13898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f62a83d9dc34126b38fca7d1f0413c3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5f62a83d9dc34126b38fca7d1f0413c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76572ad082cd49758fe05a17dfb0d686" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 43431 } } peers { permanent_uuid: "5f62a83d9dc34126b38fca7d1f0413c3" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 46461 } } peers { permanent_uuid: "6724e2a931ab42b6bd6930d2eb87bbd5" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 43521 } } }
I20260808 11:40:48.162655 13898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f62a83d9dc34126b38fca7d1f0413c3 [sys.catalog]: This master's current role is: LEADER
I20260808 11:40:48.163728 13872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76572ad082cd49758fe05a17dfb0d686 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f62a83d9dc34126b38fca7d1f0413c3. Latest consensus state: current_term: 1 leader_uuid: "5f62a83d9dc34126b38fca7d1f0413c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76572ad082cd49758fe05a17dfb0d686" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 43431 } } peers { permanent_uuid: "5f62a83d9dc34126b38fca7d1f0413c3" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 46461 } } peers { permanent_uuid: "6724e2a931ab42b6bd6930d2eb87bbd5" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 43521 } } }
I20260808 11:40:48.164005 13872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76572ad082cd49758fe05a17dfb0d686 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:40:48.164901 13848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6724e2a931ab42b6bd6930d2eb87bbd5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5f62a83d9dc34126b38fca7d1f0413c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76572ad082cd49758fe05a17dfb0d686" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 43431 } } peers { permanent_uuid: "5f62a83d9dc34126b38fca7d1f0413c3" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 46461 } } peers { permanent_uuid: "6724e2a931ab42b6bd6930d2eb87bbd5" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 43521 } } }
I20260808 11:40:48.165155 13848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6724e2a931ab42b6bd6930d2eb87bbd5 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:40:48.169690 13912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f62a83d9dc34126b38fca7d1f0413c3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5f62a83d9dc34126b38fca7d1f0413c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76572ad082cd49758fe05a17dfb0d686" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 43431 } } peers { permanent_uuid: "5f62a83d9dc34126b38fca7d1f0413c3" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 46461 } } peers { permanent_uuid: "6724e2a931ab42b6bd6930d2eb87bbd5" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 43521 } } }
I20260808 11:40:48.169971 13912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f62a83d9dc34126b38fca7d1f0413c3 [sys.catalog]: This master's current role is: LEADER
I20260808 11:40:48.171924 13872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76572ad082cd49758fe05a17dfb0d686 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5f62a83d9dc34126b38fca7d1f0413c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76572ad082cd49758fe05a17dfb0d686" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 43431 } } peers { permanent_uuid: "5f62a83d9dc34126b38fca7d1f0413c3" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 46461 } } peers { permanent_uuid: "6724e2a931ab42b6bd6930d2eb87bbd5" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 43521 } } }
I20260808 11:40:48.172158 13872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76572ad082cd49758fe05a17dfb0d686 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:40:48.174571 13915 catalog_manager.cc:1347] Generated new cluster ID: 90fd94f1eb094fc5bfcc2f6d3ef172cf
I20260808 11:40:48.174670 13915 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:40:48.194344 13915 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:40:48.196331 13915 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:40:48.218051 13915 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f62a83d9dc34126b38fca7d1f0413c3: Generated new TSK 0
I20260808 11:40:48.219162 13915 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:40:48.933341 13865 catalog_manager.cc:1259] Loaded cluster ID: 90fd94f1eb094fc5bfcc2f6d3ef172cf
I20260808 11:40:48.933458 13865 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6724e2a931ab42b6bd6930d2eb87bbd5: loading cluster ID for follower catalog manager: success
I20260808 11:40:48.936010 13865 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6724e2a931ab42b6bd6930d2eb87bbd5: acquiring CA information for follower catalog manager: success
I20260808 11:40:48.937774 13865 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6724e2a931ab42b6bd6930d2eb87bbd5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 11:40:48.994872 13891 catalog_manager.cc:1259] Loaded cluster ID: 90fd94f1eb094fc5bfcc2f6d3ef172cf
I20260808 11:40:48.994973 13891 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 76572ad082cd49758fe05a17dfb0d686: loading cluster ID for follower catalog manager: success
I20260808 11:40:48.997432 13891 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 76572ad082cd49758fe05a17dfb0d686: acquiring CA information for follower catalog manager: success
I20260808 11:40:48.999142 13891 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 76572ad082cd49758fe05a17dfb0d686: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 11:40:49.365622 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 13501
I20260808 11:40:49.366722 13599 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:40:49.479545 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 13501
I20260808 11:40:49.506628 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 13619
I20260808 11:40:49.507651 13706 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:40:49.613794 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 13619
W20260808 11:40:49.638355 13750 connection.cc:535] server connection from 127.0.0.1:53378 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 11:40:49.638669 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 13725
I20260808 11:40:49.639710 13821 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:40:49.745678 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 13725
2026-08-08T11:40:49Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4836 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:40:49 dist-test-slave-tc2s krb5kdc[13944](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:40:49 dist-test-slave-tc2s krb5kdc[13944](info): set up 2 sockets
Aug 08 11:40:49 dist-test-slave-tc2s krb5kdc[13944](info): commencing operation
krb5kdc: starting...
W20260808 11:40:50.969331 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.006s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:40:51 dist-test-slave-tc2s krb5kdc[13944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:40:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:40:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
I20260808 11:40:51.220966 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:45153
--webserver_interface=127.24.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:46387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:45153
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:40:51.546809 13966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:40:51.547072 13966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:40:51.547204 13966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:40:51.554112 13966 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:40:51.554205 13966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:40:51.554278 13966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:40:51.554358 13966 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:40:51.554417 13966 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:40:51.563812 13966 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:46387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:45153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:45153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:40:51.565004 13966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:40:51.566270 13966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:40:51.577417 13972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:51.577456 13978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:51.577497 13973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:40:51.578138 13966 server_base.cc:1034] running on GCE node
I20260808 11:40:51.578850 13966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:40:51.580127 13966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:40:51.581384 13966 hybrid_clock.cc:648] HybridClock initialized: now 1786189251581290 us; error 43 us; skew 500 ppm
I20260808 11:40:51.581703 13966 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:40:51 dist-test-slave-tc2s krb5kdc[13944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189251, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:40:51.584707 13966 init.cc:377] Logged in from keytab as kudu/127.24.76.126@KRBTEST.COM (short username kudu)
I20260808 11:40:51.586078 13966 webserver.cc:460] Webserver started at http://127.24.76.126:34443/ using document root <none> and password file <none>
I20260808 11:40:51.586683 13966 fs_manager.cc:359] Metadata directory not provided
I20260808 11:40:51.586778 13966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:40:51.587033 13966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:40:51.589792 13966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "c56c3986f6f9402b9eab490111f3f269"
format_stamp: "Formatted at 2026-08-08 11:40:51 on dist-test-slave-tc2s"
server_key: "63bd70fa928d6b8aa8fcb1dcad2cba67"
server_key_iv: "8a9757702ab6a5cd666e1381d5736ff0"
server_key_version: "encryptionkey@0"
I20260808 11:40:51.590528 13966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "c56c3986f6f9402b9eab490111f3f269"
format_stamp: "Formatted at 2026-08-08 11:40:51 on dist-test-slave-tc2s"
server_key: "63bd70fa928d6b8aa8fcb1dcad2cba67"
server_key_iv: "8a9757702ab6a5cd666e1381d5736ff0"
server_key_version: "encryptionkey@0"
I20260808 11:40:51.595422 13966 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 11:40:51.598599 13990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:51.599825 13966 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:40:51.599992 13966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "c56c3986f6f9402b9eab490111f3f269"
format_stamp: "Formatted at 2026-08-08 11:40:51 on dist-test-slave-tc2s"
server_key: "63bd70fa928d6b8aa8fcb1dcad2cba67"
server_key_iv: "8a9757702ab6a5cd666e1381d5736ff0"
server_key_version: "encryptionkey@0"
I20260808 11:40:51.600162 13966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:40:51.641038 13966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:40:51.644977 13966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:40:51.645236 13966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:40:51.661556 13966 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:45153
I20260808 11:40:51.661579 14065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:45153 every 8 connection(s)
I20260808 11:40:51.662909 13966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:40:51.666563 14069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:40:51.673054 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 13966
I20260808 11:40:51.673261 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:40:51.673933 24881 external_mini_cluster.cc:1428] Setting key 49975ad0b8a741a082d69bf68706904d
I20260808 11:40:51.676517 14069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c56c3986f6f9402b9eab490111f3f269: Bootstrap starting.
I20260808 11:40:51.679754 14069 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c56c3986f6f9402b9eab490111f3f269: Neither blocks nor log segments found. Creating new log.
I20260808 11:40:51.680986 14069 log.cc:824] T 00000000000000000000000000000000 P c56c3986f6f9402b9eab490111f3f269: Log is configured to *not* fsync() on all Append() calls
I20260808 11:40:51.683482 14069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c56c3986f6f9402b9eab490111f3f269: No bootstrap required, opened a new log
Aug 08 11:40:51 dist-test-slave-tc2s krb5kdc[13944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:40:51.690994 14069 raft_consensus.cc:348] T 00000000000000000000000000000000 P c56c3986f6f9402b9eab490111f3f269 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c56c3986f6f9402b9eab490111f3f269" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 45153 } }
I20260808 11:40:51.691318 14069 raft_consensus.cc:374] T 00000000000000000000000000000000 P c56c3986f6f9402b9eab490111f3f269 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:40:51.691406 14069 raft_consensus.cc:729] T 00000000000000000000000000000000 P c56c3986f6f9402b9eab490111f3f269 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c56c3986f6f9402b9eab490111f3f269, State: Initialized, Role: FOLLOWER
I20260808 11:40:51.691965 14069 consensus_queue.cc:260] T 00000000000000000000000000000000 P c56c3986f6f9402b9eab490111f3f269 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c56c3986f6f9402b9eab490111f3f269" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 45153 } }
I20260808 11:40:51.692245 14069 raft_consensus.cc:388] T 00000000000000000000000000000000 P c56c3986f6f9402b9eab490111f3f269 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:40:51.692373 14069 raft_consensus.cc:482] T 00000000000000000000000000000000 P c56c3986f6f9402b9eab490111f3f269 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:40:51.692570 14069 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c56c3986f6f9402b9eab490111f3f269 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:40:51.695478 14075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:51.675609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59192 (local address 127.24.76.126:45153)
0808 11:40:51.676323 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:51.676334 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:51.676373 (+ 39us) server_negotiation.cc:407] Connection header received
0808 11:40:51.678110 (+ 1737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:51.678131 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:51.678474 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:51.678780 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:51.680595 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:51.682002 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:51.683272 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:51.683968 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:51.684131 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:51.688049 (+ 3918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:51.688057 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:51.688070 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:51.688106 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:51.691023 (+ 2917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:51.691794 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:51.691803 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:51.691811 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:51.692032 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:51.692403 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:51.692408 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:51.692411 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:51.693107 (+ 696us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:51.693409 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:51.693784 (+ 375us) server_negotiation.cc:299] Negotiation successful
0808 11:40:51.694224 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":327,"threads_started":1}
I20260808 11:40:51.695727 14069 raft_consensus.cc:504] T 00000000000000000000000000000000 P c56c3986f6f9402b9eab490111f3f269 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c56c3986f6f9402b9eab490111f3f269" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 45153 } }
I20260808 11:40:51.696220 14069 leader_election.cc:302] T 00000000000000000000000000000000 P c56c3986f6f9402b9eab490111f3f269 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c56c3986f6f9402b9eab490111f3f269; no voters:
I20260808 11:40:51.696802 14069 leader_election.cc:288] T 00000000000000000000000000000000 P c56c3986f6f9402b9eab490111f3f269 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:40:51.696870 14077 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c56c3986f6f9402b9eab490111f3f269 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:40:51.698494 14077 raft_consensus.cc:686] T 00000000000000000000000000000000 P c56c3986f6f9402b9eab490111f3f269 [term 1 LEADER]: Becoming Leader. State: Replica: c56c3986f6f9402b9eab490111f3f269, State: Running, Role: LEADER
I20260808 11:40:51.698695 14069 sys_catalog.cc:564] T 00000000000000000000000000000000 P c56c3986f6f9402b9eab490111f3f269 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:40:51.699072 14077 consensus_queue.cc:237] T 00000000000000000000000000000000 P c56c3986f6f9402b9eab490111f3f269 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c56c3986f6f9402b9eab490111f3f269" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 45153 } }
I20260808 11:40:51.699813 24881 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:40:51.701596 14079 sys_catalog.cc:455] T 00000000000000000000000000000000 P c56c3986f6f9402b9eab490111f3f269 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c56c3986f6f9402b9eab490111f3f269" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c56c3986f6f9402b9eab490111f3f269" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 45153 } } }
I20260808 11:40:51.701996 14079 sys_catalog.cc:458] T 00000000000000000000000000000000 P c56c3986f6f9402b9eab490111f3f269 [sys.catalog]: This master's current role is: LEADER
I20260808 11:40:51.704041 14083 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:40:51.703998 14080 sys_catalog.cc:455] T 00000000000000000000000000000000 P c56c3986f6f9402b9eab490111f3f269 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c56c3986f6f9402b9eab490111f3f269. Latest consensus state: current_term: 1 leader_uuid: "c56c3986f6f9402b9eab490111f3f269" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c56c3986f6f9402b9eab490111f3f269" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 45153 } } }
I20260808 11:40:51.704321 14080 sys_catalog.cc:458] T 00000000000000000000000000000000 P c56c3986f6f9402b9eab490111f3f269 [sys.catalog]: This master's current role is: LEADER
I20260808 11:40:51.709143 14083 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:40:51.717525 14083 catalog_manager.cc:1347] Generated new cluster ID: 80035ff6f2df4c959b457d1d8612b6a1
I20260808 11:40:51.717633 14083 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:40:51.736035 14083 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:40:51.737672 14083 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:40:51.751142 14083 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c56c3986f6f9402b9eab490111f3f269: Generated new TSK 0
I20260808 11:40:51.751884 14083 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:40:51.760462 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 13966
I20260808 11:40:51.761562 14059 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:40:51.849537 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 13966
2026-08-08T11:40:51Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2096 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:40:51 dist-test-slave-tc2s krb5kdc[14111](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:40:51 dist-test-slave-tc2s krb5kdc[14111](info): set up 2 sockets
Aug 08 11:40:51 dist-test-slave-tc2s krb5kdc[14111](info): commencing operation
krb5kdc: starting...
W20260808 11:40:53.056305 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:40:53 dist-test-slave-tc2s krb5kdc[14111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:40:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:40:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
I20260808 11:40:53.303294 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:36973
--webserver_interface=127.24.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:34219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:36973
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:40:53.631507 14128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:40:53.631783 14128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:40:53.631880 14128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:40:53.638846 14128 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:40:53.638942 14128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:40:53.638993 14128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:40:53.639048 14128 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:40:53.639096 14128 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:40:53.648941 14128 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:34219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:36973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:36973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:40:53.650143 14128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:40:53.651319 14128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:40:53.662191 14139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:53.662142 14141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:53.662225 14137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:40:53.662931 14128 server_base.cc:1034] running on GCE node
I20260808 11:40:53.663679 14128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:40:53.664943 14128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:40:53.666136 14128 hybrid_clock.cc:648] HybridClock initialized: now 1786189253666094 us; error 26 us; skew 500 ppm
I20260808 11:40:53.666458 14128 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:40:53 dist-test-slave-tc2s krb5kdc[14111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189253, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:40:53.672350 14128 init.cc:377] Logged in from keytab as kudu/127.24.76.126@KRBTEST.COM (short username kudu)
I20260808 11:40:53.673746 14128 webserver.cc:460] Webserver started at http://127.24.76.126:33745/ using document root <none> and password file <none>
I20260808 11:40:53.674425 14128 fs_manager.cc:359] Metadata directory not provided
I20260808 11:40:53.674543 14128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:40:53.674815 14128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:40:53.677675 14128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "458b6770d36f44afb76ef710569715d2"
format_stamp: "Formatted at 2026-08-08 11:40:53 on dist-test-slave-tc2s"
server_key: "3dcafd6cb11bcf112adf724879de3f73"
server_key_iv: "d413c28bccec335c3d166ef34022840d"
server_key_version: "encryptionkey@0"
I20260808 11:40:53.678449 14128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "458b6770d36f44afb76ef710569715d2"
format_stamp: "Formatted at 2026-08-08 11:40:53 on dist-test-slave-tc2s"
server_key: "3dcafd6cb11bcf112adf724879de3f73"
server_key_iv: "d413c28bccec335c3d166ef34022840d"
server_key_version: "encryptionkey@0"
I20260808 11:40:53.683190 14128 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 11:40:53.686312 14151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:53.687484 14128 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260808 11:40:53.687637 14128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "458b6770d36f44afb76ef710569715d2"
format_stamp: "Formatted at 2026-08-08 11:40:53 on dist-test-slave-tc2s"
server_key: "3dcafd6cb11bcf112adf724879de3f73"
server_key_iv: "d413c28bccec335c3d166ef34022840d"
server_key_version: "encryptionkey@0"
I20260808 11:40:53.687804 14128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:40:53.712069 14128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:40:53.715744 14128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:40:53.716004 14128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:40:53.733053 14128 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:36973
I20260808 11:40:53.733084 14228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:36973 every 8 connection(s)
I20260808 11:40:53.734328 14128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:40:53.738070 14229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:40:53.739236 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 14128
I20260808 11:40:53.739420 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:40:53.740082 24881 external_mini_cluster.cc:1428] Setting key 17e0d7469b31e53b00f5586253f41559
I20260808 11:40:53.751889 14229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2: Bootstrap starting.
Aug 08 11:40:53 dist-test-slave-tc2s krb5kdc[14111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:40:53.755880 14229 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2: Neither blocks nor log segments found. Creating new log.
I20260808 11:40:53.757205 14229 log.cc:824] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2: Log is configured to *not* fsync() on all Append() calls
I20260808 11:40:53.760650 14229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2: No bootstrap required, opened a new log
I20260808 11:40:53.764777 14232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:53.742146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48218 (local address 127.24.76.126:36973)
0808 11:40:53.742929 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:53.742943 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:53.742983 (+ 40us) server_negotiation.cc:407] Connection header received
0808 11:40:53.744505 (+ 1522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:53.744527 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:53.744993 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:53.745372 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:53.747106 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:53.748467 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:53.750323 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:53.750983 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:53.751111 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:53.756177 (+ 5066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:53.756193 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:53.756214 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:53.756273 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:53.760121 (+ 3848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:53.760902 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:53.760910 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:53.760919 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:53.761136 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:53.761552 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:53.761557 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:53.761560 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:53.762273 (+ 713us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:53.762587 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:53.762996 (+ 409us) server_negotiation.cc:299] Negotiation successful
0808 11:40:53.763441 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":353,"threads_started":1}
I20260808 11:40:53.770012 14229 raft_consensus.cc:348] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "458b6770d36f44afb76ef710569715d2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 36973 } }
I20260808 11:40:53.770350 14229 raft_consensus.cc:374] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:40:53.770430 14229 raft_consensus.cc:729] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 458b6770d36f44afb76ef710569715d2, State: Initialized, Role: FOLLOWER
I20260808 11:40:53.770934 14229 consensus_queue.cc:260] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "458b6770d36f44afb76ef710569715d2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 36973 } }
I20260808 11:40:53.771138 14229 raft_consensus.cc:388] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:40:53.771232 14229 raft_consensus.cc:482] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:40:53.771363 14229 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:40:53.773489 14229 raft_consensus.cc:504] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "458b6770d36f44afb76ef710569715d2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 36973 } }
I20260808 11:40:53.773957 14229 leader_election.cc:302] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 458b6770d36f44afb76ef710569715d2; no voters:
I20260808 11:40:53.774545 14229 leader_election.cc:288] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:40:53.774674 14234 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:40:53.775465 14234 raft_consensus.cc:686] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [term 1 LEADER]: Becoming Leader. State: Replica: 458b6770d36f44afb76ef710569715d2, State: Running, Role: LEADER
I20260808 11:40:53.775969 14234 consensus_queue.cc:237] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "458b6770d36f44afb76ef710569715d2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 36973 } }
I20260808 11:40:53.776583 14229 sys_catalog.cc:564] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:40:53.778839 14235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "458b6770d36f44afb76ef710569715d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "458b6770d36f44afb76ef710569715d2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 36973 } } }
I20260808 11:40:53.778752 14236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 458b6770d36f44afb76ef710569715d2. Latest consensus state: current_term: 1 leader_uuid: "458b6770d36f44afb76ef710569715d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "458b6770d36f44afb76ef710569715d2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 36973 } } }
I20260808 11:40:53.779182 14235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [sys.catalog]: This master's current role is: LEADER
I20260808 11:40:53.779219 14236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [sys.catalog]: This master's current role is: LEADER
I20260808 11:40:53.784044 14243 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:40:53.788441 14243 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:40:53.795836 14243 catalog_manager.cc:1347] Generated new cluster ID: 56e1e86b3cc14593a8d52c78152b399d
I20260808 11:40:53.795946 14243 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:40:53.812413 14243 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:40:53.814294 14243 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:40:53.821660 14243 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2: Generated new TSK 0
I20260808 11:40:53.822641 14243 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab.
Entry for principal kudu/127.24.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab.
I20260808 11:40:53.889518 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.65:0
--local_ip_for_outbound_sockets=127.24.76.65
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:36973
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:34219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:40:54.216993 14266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:40:54.217278 14266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:40:54.217432 14266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:40:54.224669 14266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:40:54.224784 14266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:40:54.224934 14266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.65
I20260808 11:40:54.234342 14266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:34219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:36973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.65
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:40:54.235517 14266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:40:54.236835 14266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:40:54.249233 14275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:54.249257 14276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:54.249253 14282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:40:54.250128 14266 server_base.cc:1034] running on GCE node
I20260808 11:40:54.250712 14266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:40:54.251586 14266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:40:54.252805 14266 hybrid_clock.cc:648] HybridClock initialized: now 1786189254252757 us; error 29 us; skew 500 ppm
I20260808 11:40:54.253098 14266 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:40:54 dist-test-slave-tc2s krb5kdc[14111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189254, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:40:54.256157 14266 init.cc:377] Logged in from keytab as kudu/127.24.76.65@KRBTEST.COM (short username kudu)
I20260808 11:40:54.257520 14266 webserver.cc:460] Webserver started at http://127.24.76.65:44759/ using document root <none> and password file <none>
I20260808 11:40:54.258214 14266 fs_manager.cc:359] Metadata directory not provided
I20260808 11:40:54.258347 14266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:40:54.258620 14266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:40:54.261406 14266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/data/instance:
uuid: "d77df0c06c4b49288f5d8feddb6e9691"
format_stamp: "Formatted at 2026-08-08 11:40:54 on dist-test-slave-tc2s"
server_key: "1dd4a2a594b34009141005e2ea168904"
server_key_iv: "de79f555936eb8ac8c0d68a0a070a8eb"
server_key_version: "encryptionkey@0"
I20260808 11:40:54.262231 14266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance:
uuid: "d77df0c06c4b49288f5d8feddb6e9691"
format_stamp: "Formatted at 2026-08-08 11:40:54 on dist-test-slave-tc2s"
server_key: "1dd4a2a594b34009141005e2ea168904"
server_key_iv: "de79f555936eb8ac8c0d68a0a070a8eb"
server_key_version: "encryptionkey@0"
I20260808 11:40:54.267266 14266 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 11:40:54.270494 14294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:54.271732 14266 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:40:54.271880 14266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/wal
uuid: "d77df0c06c4b49288f5d8feddb6e9691"
format_stamp: "Formatted at 2026-08-08 11:40:54 on dist-test-slave-tc2s"
server_key: "1dd4a2a594b34009141005e2ea168904"
server_key_iv: "de79f555936eb8ac8c0d68a0a070a8eb"
server_key_version: "encryptionkey@0"
I20260808 11:40:54.272051 14266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:40:54.308306 14266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:40:54.312119 14266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:40:54.312393 14266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:40:54.313319 14266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:40:54.314664 14266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:40:54.314735 14266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:54.314860 14266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:40:54.314922 14266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:54.345839 14266 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.65:42507
I20260808 11:40:54.345903 14476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.65:42507 every 8 connection(s)
I20260808 11:40:54.347172 14266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
I20260808 11:40:54.355082 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 14266
I20260808 11:40:54.355284 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance
I20260808 11:40:54.355931 24881 external_mini_cluster.cc:1428] Setting key 37fe888fbe996a233e3a2fc8c03ca32e
Aug 08 11:40:54 dist-test-slave-tc2s krb5kdc[14111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189254, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:40:54.367395 14484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:54.350005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:45751 (local address 127.24.76.126:36973)
0808 11:40:54.350432 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:54.350437 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:54.351299 (+ 862us) server_negotiation.cc:407] Connection header received
0808 11:40:54.353088 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:54.353094 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:54.353171 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:54.353267 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:54.354777 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:54.355563 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:54.357680 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:54.358193 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:54.358294 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:54.362026 (+ 3732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:54.362031 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:54.362034 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:54.362067 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:54.364512 (+ 2445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:54.365182 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:54.365186 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:54.365189 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:54.365305 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:54.365712 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:54.365717 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:54.365719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:54.366020 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:54.366152 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:54.366815 (+ 663us) server_negotiation.cc:299] Negotiation successful
0808 11:40:54.367060 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":224,"threads_started":1}
I20260808 11:40:54.368216 14485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:54.350456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.65:45751)
0808 11:40:54.351131 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:54.351180 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:54.352830 (+ 1650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:54.353514 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:54.353521 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:54.354050 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:40:54.354602 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:54.354621 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:54.355710 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:54.355715 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:54.357535 (+ 1820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:54.357551 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:54.358417 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:54.358426 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:54.358717 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:54.359368 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:40:54.359401 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:40:54.361803 (+ 2402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:40:54.364640 (+ 2837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:54.364649 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:54.364664 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:54.365058 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:54.365412 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:54.365417 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:54.365421 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:54.365598 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:54.366139 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:40:54.366147 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:40:54.366460 (+ 313us) client_negotiation.cc:769] Sending connection context
0808 11:40:54.366681 (+ 221us) client_negotiation.cc:240] Negotiation successful
0808 11:40:54.367159 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":264,"threads_started":1}
I20260808 11:40:54.369567 14478 heartbeater.cc:347] Connected to a master server at 127.24.76.126:36973
I20260808 11:40:54.369836 14478 heartbeater.cc:464] Registering TS with master...
I20260808 11:40:54.370553 14478 heartbeater.cc:511] Master 127.24.76.126:36973 requested a full tablet report, sending...
I20260808 11:40:54.372449 14183 ts_manager.cc:194] Registered new tserver with Master: d77df0c06c4b49288f5d8feddb6e9691 (127.24.76.65:42507)
I20260808 11:40:54.373817 14183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.65@KRBTEST.COM'} at 127.24.76.65:45751
WARNING: no policy specified for kudu/127.24.76.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab.
Entry for principal kudu/127.24.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab.
I20260808 11:40:54.412536 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.66:0
--local_ip_for_outbound_sockets=127.24.76.66
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:36973
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:34219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:40:54.735519 14491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:40:54.735801 14491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:40:54.735920 14491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:40:54.742810 14491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:40:54.742899 14491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:40:54.743041 14491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.66
I20260808 11:40:54.752131 14491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:34219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:36973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.66
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:40:54.753197 14491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:40:54.754423 14491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:40:54.767308 14500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:54.767395 14498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:54.767370 14502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:40:54.768224 14491 server_base.cc:1034] running on GCE node
I20260808 11:40:54.768785 14491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:40:54.769629 14491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:40:54.770828 14491 hybrid_clock.cc:648] HybridClock initialized: now 1786189254770792 us; error 52 us; skew 500 ppm
I20260808 11:40:54.771126 14491 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:40:54 dist-test-slave-tc2s krb5kdc[14111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189254, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:40:54.774019 14491 init.cc:377] Logged in from keytab as kudu/127.24.76.66@KRBTEST.COM (short username kudu)
I20260808 11:40:54.775306 14491 webserver.cc:460] Webserver started at http://127.24.76.66:34609/ using document root <none> and password file <none>
I20260808 11:40:54.775920 14491 fs_manager.cc:359] Metadata directory not provided
I20260808 11:40:54.776039 14491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:40:54.776374 14491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:40:54.779183 14491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/data/instance:
uuid: "edff6837b14e4df1afe5b23c274ab44e"
format_stamp: "Formatted at 2026-08-08 11:40:54 on dist-test-slave-tc2s"
server_key: "0c56724677c9398dc750b305ffa0b3c7"
server_key_iv: "722b13ef4d67a0ef14f375f71f1c74b8"
server_key_version: "encryptionkey@0"
I20260808 11:40:54.779909 14491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance:
uuid: "edff6837b14e4df1afe5b23c274ab44e"
format_stamp: "Formatted at 2026-08-08 11:40:54 on dist-test-slave-tc2s"
server_key: "0c56724677c9398dc750b305ffa0b3c7"
server_key_iv: "722b13ef4d67a0ef14f375f71f1c74b8"
server_key_version: "encryptionkey@0"
I20260808 11:40:54.784821 14491 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 11:40:54.788105 14515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:54.789299 14491 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 11:40:54.789446 14491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/wal
uuid: "edff6837b14e4df1afe5b23c274ab44e"
format_stamp: "Formatted at 2026-08-08 11:40:54 on dist-test-slave-tc2s"
server_key: "0c56724677c9398dc750b305ffa0b3c7"
server_key_iv: "722b13ef4d67a0ef14f375f71f1c74b8"
server_key_version: "encryptionkey@0"
I20260808 11:40:54.789626 14491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:40:54.811700 14491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:40:54.815975 14491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:40:54.816218 14491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:40:54.817082 14491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:40:54.818357 14491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:40:54.818431 14491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:54.818517 14491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:40:54.818559 14491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:54.848604 14491 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.66:43777
I20260808 11:40:54.848649 14676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.66:43777 every 8 connection(s)
I20260808 11:40:54.849841 14491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
I20260808 11:40:54.857568 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 14491
I20260808 11:40:54.857738 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance
I20260808 11:40:54.858227 24881 external_mini_cluster.cc:1428] Setting key 267c586c5de313a7ed7a992fd58a99ed
Aug 08 11:40:54 dist-test-slave-tc2s krb5kdc[14111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189254, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:40:54.868541 14484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:54.852304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:54155 (local address 127.24.76.126:36973)
0808 11:40:54.852477 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:54.852482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:54.853614 (+ 1132us) server_negotiation.cc:407] Connection header received
0808 11:40:54.855418 (+ 1804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:54.855424 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:54.855519 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:54.855606 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:54.857013 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:54.857788 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:54.859470 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:54.859916 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:54.860026 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:54.863272 (+ 3246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:54.863277 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:54.863281 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:54.863311 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:54.865642 (+ 2331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:54.866381 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:54.866387 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:54.866389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:54.866521 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:54.866919 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:54.866923 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:54.866925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:54.867182 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:54.867312 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:54.867980 (+ 668us) server_negotiation.cc:299] Negotiation successful
0808 11:40:54.868201 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 11:40:54.869410 14680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:54.852770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.66:54155)
0808 11:40:54.853463 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:54.853509 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:54.855162 (+ 1653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:54.855831 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:54.855839 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:54.856324 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:40:54.856848 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:54.856864 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:54.857943 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:54.857947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:54.859339 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:54.859352 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:54.860138 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:54.860146 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:54.860360 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:54.860949 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:40:54.860980 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:40:54.863059 (+ 2079us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:40:54.865767 (+ 2708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:54.865775 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:54.865789 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:54.866261 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:54.866621 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:54.866626 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:54.866630 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:54.866808 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:54.867322 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:40:54.867330 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:40:54.867630 (+ 300us) client_negotiation.cc:769] Sending connection context
0808 11:40:54.867848 (+ 218us) client_negotiation.cc:240] Negotiation successful
0808 11:40:54.868332 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":269,"threads_started":1}
I20260808 11:40:54.870702 14677 heartbeater.cc:347] Connected to a master server at 127.24.76.126:36973
I20260808 11:40:54.871018 14677 heartbeater.cc:464] Registering TS with master...
I20260808 11:40:54.871784 14677 heartbeater.cc:511] Master 127.24.76.126:36973 requested a full tablet report, sending...
I20260808 11:40:54.873180 14183 ts_manager.cc:194] Registered new tserver with Master: edff6837b14e4df1afe5b23c274ab44e (127.24.76.66:43777)
I20260808 11:40:54.874027 14183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.66@KRBTEST.COM'} at 127.24.76.66:54155
WARNING: no policy specified for kudu/127.24.76.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab.
Entry for principal kudu/127.24.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab.
I20260808 11:40:54.915192 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.67:0
--local_ip_for_outbound_sockets=127.24.76.67
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:36973
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:34219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:40:55.237051 14684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:40:55.237341 14684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:40:55.237447 14684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:40:55.244578 14684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:40:55.244676 14684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:40:55.244792 14684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.67
I20260808 11:40:55.254020 14684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:34219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:36973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.67
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:40:55.255127 14684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:40:55.256388 14684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:40:55.269001 14697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:55.269037 14695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:55.269086 14694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:40:55.269950 14684 server_base.cc:1034] running on GCE node
I20260808 11:40:55.270522 14684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:40:55.271366 14684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:40:55.272563 14684 hybrid_clock.cc:648] HybridClock initialized: now 1786189255272513 us; error 44 us; skew 500 ppm
I20260808 11:40:55.272845 14684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:40:55 dist-test-slave-tc2s krb5kdc[14111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189255, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:40:55.275840 14684 init.cc:377] Logged in from keytab as kudu/127.24.76.67@KRBTEST.COM (short username kudu)
I20260808 11:40:55.277230 14684 webserver.cc:460] Webserver started at http://127.24.76.67:34563/ using document root <none> and password file <none>
I20260808 11:40:55.277879 14684 fs_manager.cc:359] Metadata directory not provided
I20260808 11:40:55.278035 14684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:40:55.278316 14684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:40:55.281141 14684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/data/instance:
uuid: "3d02a3b7b1384997b48d3255f708864c"
format_stamp: "Formatted at 2026-08-08 11:40:55 on dist-test-slave-tc2s"
server_key: "9e705ee2ee6732036a0f3465c60d9873"
server_key_iv: "611ad4bceec41c52e7f0cdd1e38a1bc0"
server_key_version: "encryptionkey@0"
I20260808 11:40:55.281947 14684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance:
uuid: "3d02a3b7b1384997b48d3255f708864c"
format_stamp: "Formatted at 2026-08-08 11:40:55 on dist-test-slave-tc2s"
server_key: "9e705ee2ee6732036a0f3465c60d9873"
server_key_iv: "611ad4bceec41c52e7f0cdd1e38a1bc0"
server_key_version: "encryptionkey@0"
I20260808 11:40:55.286988 14684 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 11:40:55.290460 14709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:55.291710 14684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260808 11:40:55.291877 14684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/wal
uuid: "3d02a3b7b1384997b48d3255f708864c"
format_stamp: "Formatted at 2026-08-08 11:40:55 on dist-test-slave-tc2s"
server_key: "9e705ee2ee6732036a0f3465c60d9873"
server_key_iv: "611ad4bceec41c52e7f0cdd1e38a1bc0"
server_key_version: "encryptionkey@0"
I20260808 11:40:55.292049 14684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:40:55.303182 14684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:40:55.306725 14684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:40:55.306978 14684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:40:55.307847 14684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:40:55.309100 14684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:40:55.309180 14684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:55.309297 14684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:40:55.309351 14684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:55.339306 14684 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.67:35923
I20260808 11:40:55.339354 14883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.67:35923 every 8 connection(s)
I20260808 11:40:55.340543 14684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
I20260808 11:40:55.347738 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 14684
I20260808 11:40:55.347929 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance
I20260808 11:40:55.348377 24881 external_mini_cluster.cc:1428] Setting key b45a74c8c44d182940251e4fec27b259
Aug 08 11:40:55 dist-test-slave-tc2s krb5kdc[14111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189255, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:40:55.359401 14484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:55.342991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:36041 (local address 127.24.76.126:36973)
0808 11:40:55.343160 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:55.343165 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:55.344210 (+ 1045us) server_negotiation.cc:407] Connection header received
0808 11:40:55.346030 (+ 1820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:55.346037 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:55.346147 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:55.346284 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:55.347826 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:55.349013 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:55.350958 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:55.351478 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:55.351578 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:55.354696 (+ 3118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:55.354701 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:55.354704 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:55.354735 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:55.356985 (+ 2250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:55.357602 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:55.357605 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:55.357608 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:55.357727 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:55.358113 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:55.358116 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:55.358118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:55.358363 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:55.358469 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:55.359008 (+ 539us) server_negotiation.cc:299] Negotiation successful
0808 11:40:55.359158 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 11:40:55.360323 14891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:55.343341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.67:36041)
0808 11:40:55.344021 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:55.344099 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:55.345740 (+ 1641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:55.346497 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:55.346505 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:55.346977 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:40:55.347617 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:55.347642 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:55.349173 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:55.349179 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:55.350818 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:55.350835 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:55.351693 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:55.351698 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:55.351882 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:55.352438 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:40:55.352459 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:40:55.354517 (+ 2058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:40:55.357109 (+ 2592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:55.357115 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:55.357127 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:55.357517 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:55.357800 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:55.357803 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:55.357805 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:55.358028 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:55.358450 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:40:55.358455 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:40:55.358718 (+ 263us) client_negotiation.cc:769] Sending connection context
0808 11:40:55.358924 (+ 206us) client_negotiation.cc:240] Negotiation successful
0808 11:40:55.359282 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":262,"threads_started":1}
I20260808 11:40:55.361567 14884 heartbeater.cc:347] Connected to a master server at 127.24.76.126:36973
I20260808 11:40:55.361907 14884 heartbeater.cc:464] Registering TS with master...
I20260808 11:40:55.362766 14884 heartbeater.cc:511] Master 127.24.76.126:36973 requested a full tablet report, sending...
I20260808 11:40:55.364205 14183 ts_manager.cc:194] Registered new tserver with Master: 3d02a3b7b1384997b48d3255f708864c (127.24.76.67:35923)
I20260808 11:40:55.364944 14183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.67@KRBTEST.COM'} at 127.24.76.67:36041
I20260808 11:40:55.375368 24881 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:40:55.376647 14478 heartbeater.cc:503] Master 127.24.76.126:36973 was elected leader, sending a full tablet report...
I20260808 11:40:55.391608 14484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:55.380183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48226 (local address 127.24.76.126:36973)
0808 11:40:55.380329 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:55.380333 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:55.380791 (+ 458us) server_negotiation.cc:407] Connection header received
0808 11:40:55.380917 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:55.380920 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:55.380974 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:55.381059 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:55.382384 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:55.383077 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:55.384317 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:55.384696 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:55.384789 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:55.386919 (+ 2130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:55.386924 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:55.386928 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:55.386956 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:55.389332 (+ 2376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:55.389998 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:55.390002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:55.390004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:55.390125 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:55.390435 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:55.390438 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:55.390440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:55.390739 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:55.390923 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:55.391175 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 11:40:55.391326 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 11:40:55.395192 14183 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48226:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:40:55.401831 14183 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:40:55.437466 14917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:55.419007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60762 (local address 127.24.76.66:43777)
0808 11:40:55.430254 (+ 11247us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:55.430261 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:55.430284 (+ 23us) server_negotiation.cc:407] Connection header received
0808 11:40:55.430366 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:55.430370 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:55.430564 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:55.430739 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:40:55.431409 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:55.432943 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:55.435453 (+ 2510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:55.436347 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:55.436469 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:55.436675 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:55.437018 (+ 343us) server_negotiation.cc:299] Negotiation successful
0808 11:40:55.437175 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11112,"thread_start_us":11019,"threads_started":1}
I20260808 11:40:55.437525 14911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:55.418505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:43777 (local address 127.0.0.1:60762)
0808 11:40:55.429379 (+ 10874us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:55.429399 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:55.429534 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:55.430762 (+ 1228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:55.430768 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:55.430798 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:40:55.431217 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:55.431231 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:55.433122 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:55.433127 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:55.435296 (+ 2169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:55.435313 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:55.436571 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:55.436577 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:55.436793 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:55.436840 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 11:40:55.436957 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 11:40:55.437139 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10696,"mutex_wait_us":79,"thread_start_us":10539,"threads_started":1}
I20260808 11:40:55.439407 14904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:55.416382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:35923 (local address 127.0.0.1:60734)
0808 11:40:55.423283 (+ 6901us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:55.423320 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:55.423504 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:55.431980 (+ 8476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:55.431985 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:55.432011 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:40:55.432437 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:55.432452 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:55.434773 (+ 2321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:55.434777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:55.436938 (+ 2161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:55.436951 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:55.438732 (+ 1781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:55.438739 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:55.438917 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:55.438947 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 11:40:55.439026 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 11:40:55.439145 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6662,"mutex_wait_us":2,"thread_start_us":807,"threads_started":1}
I20260808 11:40:55.440280 14906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:55.416711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:42507 (local address 127.0.0.1:49226)
0808 11:40:55.428552 (+ 11841us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:55.428571 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:55.428688 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:55.432359 (+ 3671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:55.432364 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:55.432394 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:40:55.432829 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:55.432844 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:55.434401 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:55.434406 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:55.436541 (+ 2135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:55.436560 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:55.438719 (+ 2159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:55.438726 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:55.439708 (+ 982us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:55.439731 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 11:40:55.439819 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 11:40:55.439936 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11698,"mutex_wait_us":789,"thread_start_us":12047,"threads_started":1}
I20260808 11:40:55.440378 14910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:55.417121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49226 (local address 127.24.76.65:42507)
0808 11:40:55.431752 (+ 14631us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:55.431758 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:55.431793 (+ 35us) server_negotiation.cc:407] Connection header received
0808 11:40:55.431890 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:55.431896 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:55.432128 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:55.433013 (+ 885us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:40:55.433219 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:55.434263 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:55.436725 (+ 2462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:55.438464 (+ 1739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:55.438847 (+ 383us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:55.439087 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:55.439875 (+ 788us) server_negotiation.cc:299] Negotiation successful
0808 11:40:55.440037 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14488,"thread_start_us":14329,"threads_started":1}
I20260808 11:40:55.441120 14908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:55.416905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60734 (local address 127.24.76.67:35923)
0808 11:40:55.431398 (+ 14493us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:55.431406 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:55.431437 (+ 31us) server_negotiation.cc:407] Connection header received
0808 11:40:55.431547 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:55.431553 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:55.431774 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:55.432656 (+ 882us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:40:55.433006 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:55.434643 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:55.437116 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:55.438464 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:55.440493 (+ 2029us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:55.440705 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:55.440761 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 11:40:55.440877 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14329,"thread_start_us":14159,"threads_started":1}
I20260808 11:40:55.441303 14588 tablet_service.cc:1467] Processing CreateTablet for tablet 366fafae0c1d4f8b8677f64b062e9c2e (DEFAULT_TABLE table=test-table [id=3be7658c1c6f413fbd0bb6e900dbd9f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:40:55.442713 14588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 366fafae0c1d4f8b8677f64b062e9c2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:40:55.444523 14781 tablet_service.cc:1467] Processing CreateTablet for tablet 366fafae0c1d4f8b8677f64b062e9c2e (DEFAULT_TABLE table=test-table [id=3be7658c1c6f413fbd0bb6e900dbd9f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:40:55.444527 14370 tablet_service.cc:1467] Processing CreateTablet for tablet 366fafae0c1d4f8b8677f64b062e9c2e (DEFAULT_TABLE table=test-table [id=3be7658c1c6f413fbd0bb6e900dbd9f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:40:55.445865 14370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 366fafae0c1d4f8b8677f64b062e9c2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:40:55.445886 14781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 366fafae0c1d4f8b8677f64b062e9c2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:40:55.454524 14922 tablet_bootstrap.cc:492] T 366fafae0c1d4f8b8677f64b062e9c2e P edff6837b14e4df1afe5b23c274ab44e: Bootstrap starting.
I20260808 11:40:55.457229 14922 tablet_bootstrap.cc:654] T 366fafae0c1d4f8b8677f64b062e9c2e P edff6837b14e4df1afe5b23c274ab44e: Neither blocks nor log segments found. Creating new log.
I20260808 11:40:55.458240 14922 log.cc:824] T 366fafae0c1d4f8b8677f64b062e9c2e P edff6837b14e4df1afe5b23c274ab44e: Log is configured to *not* fsync() on all Append() calls
I20260808 11:40:55.459977 14928 tablet_bootstrap.cc:492] T 366fafae0c1d4f8b8677f64b062e9c2e P d77df0c06c4b49288f5d8feddb6e9691: Bootstrap starting.
I20260808 11:40:55.461135 14922 tablet_bootstrap.cc:492] T 366fafae0c1d4f8b8677f64b062e9c2e P edff6837b14e4df1afe5b23c274ab44e: No bootstrap required, opened a new log
I20260808 11:40:55.461403 14922 ts_tablet_manager.cc:1397] T 366fafae0c1d4f8b8677f64b062e9c2e P edff6837b14e4df1afe5b23c274ab44e: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.004s
I20260808 11:40:55.462576 14925 tablet_bootstrap.cc:492] T 366fafae0c1d4f8b8677f64b062e9c2e P 3d02a3b7b1384997b48d3255f708864c: Bootstrap starting.
I20260808 11:40:55.464463 14928 tablet_bootstrap.cc:654] T 366fafae0c1d4f8b8677f64b062e9c2e P d77df0c06c4b49288f5d8feddb6e9691: Neither blocks nor log segments found. Creating new log.
I20260808 11:40:55.465677 14925 tablet_bootstrap.cc:654] T 366fafae0c1d4f8b8677f64b062e9c2e P 3d02a3b7b1384997b48d3255f708864c: Neither blocks nor log segments found. Creating new log.
I20260808 11:40:55.465700 14928 log.cc:824] T 366fafae0c1d4f8b8677f64b062e9c2e P d77df0c06c4b49288f5d8feddb6e9691: Log is configured to *not* fsync() on all Append() calls
I20260808 11:40:55.466784 14925 log.cc:824] T 366fafae0c1d4f8b8677f64b062e9c2e P 3d02a3b7b1384997b48d3255f708864c: Log is configured to *not* fsync() on all Append() calls
I20260808 11:40:55.468729 14928 tablet_bootstrap.cc:492] T 366fafae0c1d4f8b8677f64b062e9c2e P d77df0c06c4b49288f5d8feddb6e9691: No bootstrap required, opened a new log
I20260808 11:40:55.468982 14928 ts_tablet_manager.cc:1397] T 366fafae0c1d4f8b8677f64b062e9c2e P d77df0c06c4b49288f5d8feddb6e9691: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260808 11:40:55.469180 14925 tablet_bootstrap.cc:492] T 366fafae0c1d4f8b8677f64b062e9c2e P 3d02a3b7b1384997b48d3255f708864c: No bootstrap required, opened a new log
I20260808 11:40:55.469403 14925 ts_tablet_manager.cc:1397] T 366fafae0c1d4f8b8677f64b062e9c2e P 3d02a3b7b1384997b48d3255f708864c: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 11:40:55.471725 14922 raft_consensus.cc:348] T 366fafae0c1d4f8b8677f64b062e9c2e P edff6837b14e4df1afe5b23c274ab44e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d02a3b7b1384997b48d3255f708864c" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 35923 } } peers { permanent_uuid: "d77df0c06c4b49288f5d8feddb6e9691" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 42507 } } peers { permanent_uuid: "edff6837b14e4df1afe5b23c274ab44e" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 43777 } }
I20260808 11:40:55.472265 14922 raft_consensus.cc:374] T 366fafae0c1d4f8b8677f64b062e9c2e P edff6837b14e4df1afe5b23c274ab44e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:40:55.472378 14922 raft_consensus.cc:729] T 366fafae0c1d4f8b8677f64b062e9c2e P edff6837b14e4df1afe5b23c274ab44e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edff6837b14e4df1afe5b23c274ab44e, State: Initialized, Role: FOLLOWER
I20260808 11:40:55.472971 14922 consensus_queue.cc:260] T 366fafae0c1d4f8b8677f64b062e9c2e P edff6837b14e4df1afe5b23c274ab44e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d02a3b7b1384997b48d3255f708864c" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 35923 } } peers { permanent_uuid: "d77df0c06c4b49288f5d8feddb6e9691" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 42507 } } peers { permanent_uuid: "edff6837b14e4df1afe5b23c274ab44e" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 43777 } }
I20260808 11:40:55.474076 14677 heartbeater.cc:503] Master 127.24.76.126:36973 was elected leader, sending a full tablet report...
I20260808 11:40:55.474647 14922 ts_tablet_manager.cc:1428] T 366fafae0c1d4f8b8677f64b062e9c2e P edff6837b14e4df1afe5b23c274ab44e: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
I20260808 11:40:55.477413 14925 raft_consensus.cc:348] T 366fafae0c1d4f8b8677f64b062e9c2e P 3d02a3b7b1384997b48d3255f708864c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d02a3b7b1384997b48d3255f708864c" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 35923 } } peers { permanent_uuid: "d77df0c06c4b49288f5d8feddb6e9691" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 42507 } } peers { permanent_uuid: "edff6837b14e4df1afe5b23c274ab44e" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 43777 } }
I20260808 11:40:55.477769 14925 raft_consensus.cc:374] T 366fafae0c1d4f8b8677f64b062e9c2e P 3d02a3b7b1384997b48d3255f708864c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:40:55.477870 14925 raft_consensus.cc:729] T 366fafae0c1d4f8b8677f64b062e9c2e P 3d02a3b7b1384997b48d3255f708864c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d02a3b7b1384997b48d3255f708864c, State: Initialized, Role: FOLLOWER
I20260808 11:40:55.478617 14925 consensus_queue.cc:260] T 366fafae0c1d4f8b8677f64b062e9c2e P 3d02a3b7b1384997b48d3255f708864c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d02a3b7b1384997b48d3255f708864c" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 35923 } } peers { permanent_uuid: "d77df0c06c4b49288f5d8feddb6e9691" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 42507 } } peers { permanent_uuid: "edff6837b14e4df1afe5b23c274ab44e" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 43777 } }
I20260808 11:40:55.478826 14928 raft_consensus.cc:348] T 366fafae0c1d4f8b8677f64b062e9c2e P d77df0c06c4b49288f5d8feddb6e9691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d02a3b7b1384997b48d3255f708864c" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 35923 } } peers { permanent_uuid: "d77df0c06c4b49288f5d8feddb6e9691" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 42507 } } peers { permanent_uuid: "edff6837b14e4df1afe5b23c274ab44e" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 43777 } }
I20260808 11:40:55.479290 14928 raft_consensus.cc:374] T 366fafae0c1d4f8b8677f64b062e9c2e P d77df0c06c4b49288f5d8feddb6e9691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:40:55.479429 14928 raft_consensus.cc:729] T 366fafae0c1d4f8b8677f64b062e9c2e P d77df0c06c4b49288f5d8feddb6e9691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d77df0c06c4b49288f5d8feddb6e9691, State: Initialized, Role: FOLLOWER
I20260808 11:40:55.479769 14884 heartbeater.cc:503] Master 127.24.76.126:36973 was elected leader, sending a full tablet report...
I20260808 11:40:55.480479 14925 ts_tablet_manager.cc:1428] T 366fafae0c1d4f8b8677f64b062e9c2e P 3d02a3b7b1384997b48d3255f708864c: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260808 11:40:55.480901 14928 consensus_queue.cc:260] T 366fafae0c1d4f8b8677f64b062e9c2e P d77df0c06c4b49288f5d8feddb6e9691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d02a3b7b1384997b48d3255f708864c" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 35923 } } peers { permanent_uuid: "d77df0c06c4b49288f5d8feddb6e9691" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 42507 } } peers { permanent_uuid: "edff6837b14e4df1afe5b23c274ab44e" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 43777 } }
I20260808 11:40:55.482429 14928 ts_tablet_manager.cc:1428] T 366fafae0c1d4f8b8677f64b062e9c2e P d77df0c06c4b49288f5d8feddb6e9691: Time spent starting tablet: real 0.013s user 0.010s sys 0.004s
I20260808 11:40:55.531785 14932 raft_consensus.cc:482] T 366fafae0c1d4f8b8677f64b062e9c2e P 3d02a3b7b1384997b48d3255f708864c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:40:55.532050 14932 raft_consensus.cc:504] T 366fafae0c1d4f8b8677f64b062e9c2e P 3d02a3b7b1384997b48d3255f708864c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d02a3b7b1384997b48d3255f708864c" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 35923 } } peers { permanent_uuid: "d77df0c06c4b49288f5d8feddb6e9691" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 42507 } } peers { permanent_uuid: "edff6837b14e4df1afe5b23c274ab44e" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 43777 } }
I20260808 11:40:55.533756 14932 leader_election.cc:288] T 366fafae0c1d4f8b8677f64b062e9c2e P 3d02a3b7b1384997b48d3255f708864c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d77df0c06c4b49288f5d8feddb6e9691 (127.24.76.65:42507), edff6837b14e4df1afe5b23c274ab44e (127.24.76.66:43777)
I20260808 11:40:55.540537 14891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:55.534201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:42507 (local address 127.24.76.67:38393)
0808 11:40:55.534438 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:55.534462 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:55.534608 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:55.535093 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:55.535103 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:55.535132 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:40:55.535333 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:55.535345 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:55.536609 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:55.536612 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:55.538523 (+ 1911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:55.538540 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:55.539907 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:55.539911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:55.540052 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:55.540076 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 11:40:55.540146 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 11:40:55.540235 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260808 11:40:55.540568 14910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:55.534232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:38393 (local address 127.24.76.65:42507)
0808 11:40:55.534433 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:55.534439 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:55.534531 (+ 92us) server_negotiation.cc:407] Connection header received
0808 11:40:55.534851 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:55.534857 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:55.534941 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:55.535037 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:40:55.535471 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:55.536488 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:55.538693 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:55.539760 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:55.539931 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:55.540006 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:55.540195 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 11:40:55.540340 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 11:40:55.541122 14917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:55.534644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:36765 (local address 127.24.76.66:43777)
0808 11:40:55.534890 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:55.534896 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:55.535214 (+ 318us) server_negotiation.cc:407] Connection header received
0808 11:40:55.535378 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:55.535382 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:55.535455 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:55.535542 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:40:55.535979 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:55.536860 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:55.539029 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:55.540172 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:55.540382 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:55.540454 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:55.540749 (+ 295us) server_negotiation.cc:299] Negotiation successful
0808 11:40:55.540880 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260808 11:40:55.541667 14935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:55.534347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:43777 (local address 127.24.76.67:36765)
0808 11:40:55.535111 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:55.535127 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:55.535242 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:55.535570 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:55.535573 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:55.535593 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:40:55.535838 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:55.535845 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:55.537037 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:55.537042 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:55.538894 (+ 1852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:55.538905 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:55.540381 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:55.540387 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:55.540559 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:55.540590 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 11:40:55.541284 (+ 694us) client_negotiation.cc:240] Negotiation successful
0808 11:40:55.541407 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"mutex_wait_us":1,"thread_start_us":410,"threads_started":1}
I20260808 11:40:55.541616 14403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "366fafae0c1d4f8b8677f64b062e9c2e" candidate_uuid: "3d02a3b7b1384997b48d3255f708864c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d77df0c06c4b49288f5d8feddb6e9691" is_pre_election: true
I20260808 11:40:55.542218 14403 raft_consensus.cc:2454] T 366fafae0c1d4f8b8677f64b062e9c2e P d77df0c06c4b49288f5d8feddb6e9691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d02a3b7b1384997b48d3255f708864c in term 0.
I20260808 11:40:55.542608 14614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "366fafae0c1d4f8b8677f64b062e9c2e" candidate_uuid: "3d02a3b7b1384997b48d3255f708864c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edff6837b14e4df1afe5b23c274ab44e" is_pre_election: true
I20260808 11:40:55.542917 14720 leader_election.cc:302] T 366fafae0c1d4f8b8677f64b062e9c2e P 3d02a3b7b1384997b48d3255f708864c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d02a3b7b1384997b48d3255f708864c, d77df0c06c4b49288f5d8feddb6e9691; no voters:
I20260808 11:40:55.543167 14614 raft_consensus.cc:2454] T 366fafae0c1d4f8b8677f64b062e9c2e P edff6837b14e4df1afe5b23c274ab44e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d02a3b7b1384997b48d3255f708864c in term 0.
I20260808 11:40:55.543325 14932 raft_consensus.cc:2781] T 366fafae0c1d4f8b8677f64b062e9c2e P 3d02a3b7b1384997b48d3255f708864c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:40:55.543560 14932 raft_consensus.cc:482] T 366fafae0c1d4f8b8677f64b062e9c2e P 3d02a3b7b1384997b48d3255f708864c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:40:55.543686 14932 raft_consensus.cc:3037] T 366fafae0c1d4f8b8677f64b062e9c2e P 3d02a3b7b1384997b48d3255f708864c [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:40:55.545971 14932 raft_consensus.cc:504] T 366fafae0c1d4f8b8677f64b062e9c2e P 3d02a3b7b1384997b48d3255f708864c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d02a3b7b1384997b48d3255f708864c" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 35923 } } peers { permanent_uuid: "d77df0c06c4b49288f5d8feddb6e9691" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 42507 } } peers { permanent_uuid: "edff6837b14e4df1afe5b23c274ab44e" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 43777 } }
I20260808 11:40:55.546530 14932 leader_election.cc:288] T 366fafae0c1d4f8b8677f64b062e9c2e P 3d02a3b7b1384997b48d3255f708864c [CANDIDATE]: Term 1 election: Requested vote from peers d77df0c06c4b49288f5d8feddb6e9691 (127.24.76.65:42507), edff6837b14e4df1afe5b23c274ab44e (127.24.76.66:43777)
I20260808 11:40:55.547021 14614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "366fafae0c1d4f8b8677f64b062e9c2e" candidate_uuid: "3d02a3b7b1384997b48d3255f708864c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edff6837b14e4df1afe5b23c274ab44e"
I20260808 11:40:55.547021 14403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "366fafae0c1d4f8b8677f64b062e9c2e" candidate_uuid: "3d02a3b7b1384997b48d3255f708864c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d77df0c06c4b49288f5d8feddb6e9691"
I20260808 11:40:55.547226 14614 raft_consensus.cc:3037] T 366fafae0c1d4f8b8677f64b062e9c2e P edff6837b14e4df1afe5b23c274ab44e [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:40:55.547226 14403 raft_consensus.cc:3037] T 366fafae0c1d4f8b8677f64b062e9c2e P d77df0c06c4b49288f5d8feddb6e9691 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:40:55.550387 14403 raft_consensus.cc:2454] T 366fafae0c1d4f8b8677f64b062e9c2e P d77df0c06c4b49288f5d8feddb6e9691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d02a3b7b1384997b48d3255f708864c in term 1.
I20260808 11:40:55.550385 14614 raft_consensus.cc:2454] T 366fafae0c1d4f8b8677f64b062e9c2e P edff6837b14e4df1afe5b23c274ab44e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d02a3b7b1384997b48d3255f708864c in term 1.
I20260808 11:40:55.550789 14720 leader_election.cc:302] T 366fafae0c1d4f8b8677f64b062e9c2e P 3d02a3b7b1384997b48d3255f708864c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d02a3b7b1384997b48d3255f708864c, d77df0c06c4b49288f5d8feddb6e9691; no voters:
I20260808 11:40:55.551074 14932 raft_consensus.cc:2781] T 366fafae0c1d4f8b8677f64b062e9c2e P 3d02a3b7b1384997b48d3255f708864c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:40:55.551501 14932 raft_consensus.cc:686] T 366fafae0c1d4f8b8677f64b062e9c2e P 3d02a3b7b1384997b48d3255f708864c [term 1 LEADER]: Becoming Leader. State: Replica: 3d02a3b7b1384997b48d3255f708864c, State: Running, Role: LEADER
I20260808 11:40:55.551934 14932 consensus_queue.cc:237] T 366fafae0c1d4f8b8677f64b062e9c2e P 3d02a3b7b1384997b48d3255f708864c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d02a3b7b1384997b48d3255f708864c" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 35923 } } peers { permanent_uuid: "d77df0c06c4b49288f5d8feddb6e9691" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 42507 } } peers { permanent_uuid: "edff6837b14e4df1afe5b23c274ab44e" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 43777 } }
I20260808 11:40:55.556589 14181 catalog_manager.cc:5495] T 366fafae0c1d4f8b8677f64b062e9c2e P 3d02a3b7b1384997b48d3255f708864c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d02a3b7b1384997b48d3255f708864c (127.24.76.67). New cstate: current_term: 1 leader_uuid: "3d02a3b7b1384997b48d3255f708864c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d02a3b7b1384997b48d3255f708864c" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 35923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d77df0c06c4b49288f5d8feddb6e9691" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 42507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "edff6837b14e4df1afe5b23c274ab44e" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 43777 } health_report { overall_health: UNKNOWN } } }
I20260808 11:40:55.579288 14908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:55.573966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60740 (local address 127.24.76.67:35923)
0808 11:40:55.574150 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:55.574156 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:55.574175 (+ 19us) server_negotiation.cc:407] Connection header received
0808 11:40:55.574342 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:55.574346 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:55.574419 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:55.574551 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:40:55.574965 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:55.575710 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:55.577206 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:55.577603 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:55.577709 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:55.578281 (+ 572us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:40:55.578753 (+ 472us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:40:55.578881 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:55.578974 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 11:40:55.579052 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 11:40:55.586711 14614 raft_consensus.cc:1261] T 366fafae0c1d4f8b8677f64b062e9c2e P edff6837b14e4df1afe5b23c274ab44e [term 1 FOLLOWER]: Refusing update from remote peer 3d02a3b7b1384997b48d3255f708864c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:40:55.586755 14403 raft_consensus.cc:1261] T 366fafae0c1d4f8b8677f64b062e9c2e P d77df0c06c4b49288f5d8feddb6e9691 [term 1 FOLLOWER]: Refusing update from remote peer 3d02a3b7b1384997b48d3255f708864c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:40:55.587641 14932 consensus_queue.cc:1035] T 366fafae0c1d4f8b8677f64b062e9c2e P 3d02a3b7b1384997b48d3255f708864c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d77df0c06c4b49288f5d8feddb6e9691" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 42507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:40:55.587909 14936 consensus_queue.cc:1035] T 366fafae0c1d4f8b8677f64b062e9c2e P 3d02a3b7b1384997b48d3255f708864c [LEADER]: Connected to new peer: Peer: permanent_uuid: "edff6837b14e4df1afe5b23c274ab44e" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 43777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 11:40:55.591945 14885 tablet.cc:2369] T 366fafae0c1d4f8b8677f64b062e9c2e P 3d02a3b7b1384997b48d3255f708864c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:40:55.598168 14939 mvcc.cc:205] Tried to move back new op lower bound from 7316231190874435584 to 7316231190744227840. Current Snapshot: MvccSnapshot[applied={T|T < 7316231190874435584}]
I20260808 11:40:55.600128 14940 mvcc.cc:205] Tried to move back new op lower bound from 7316231190874435584 to 7316231190744227840. Current Snapshot: MvccSnapshot[applied={T|T < 7316231190874435584}]
W20260808 11:40:55.601614 14679 tablet.cc:2369] T 366fafae0c1d4f8b8677f64b062e9c2e P edff6837b14e4df1afe5b23c274ab44e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:40:55.613487 14181 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:40:55.613729 14181 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:40:55.616916 14181 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2: Sending DeleteTablet for 3 replicas of tablet 366fafae0c1d4f8b8677f64b062e9c2e
I20260808 11:40:55.617964 14781 tablet_service.cc:1514] Processing DeleteTablet for tablet 366fafae0c1d4f8b8677f64b062e9c2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:40:55 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:60734
I20260808 11:40:55.618031 14370 tablet_service.cc:1514] Processing DeleteTablet for tablet 366fafae0c1d4f8b8677f64b062e9c2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:40:55 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:49226
I20260808 11:40:55.618170 14588 tablet_service.cc:1514] Processing DeleteTablet for tablet 366fafae0c1d4f8b8677f64b062e9c2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:40:55 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:60762
I20260808 11:40:55.619041 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 14266
I20260808 11:40:55.630438 14956 tablet_replica.cc:330] T 366fafae0c1d4f8b8677f64b062e9c2e P edff6837b14e4df1afe5b23c274ab44e: stopping tablet replica
I20260808 11:40:55.631017 14954 tablet_replica.cc:330] T 366fafae0c1d4f8b8677f64b062e9c2e P d77df0c06c4b49288f5d8feddb6e9691: stopping tablet replica
I20260808 11:40:55.631605 14954 raft_consensus.cc:2229] T 366fafae0c1d4f8b8677f64b062e9c2e P d77df0c06c4b49288f5d8feddb6e9691 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:40:55.631929 14953 tablet_replica.cc:330] T 366fafae0c1d4f8b8677f64b062e9c2e P 3d02a3b7b1384997b48d3255f708864c: stopping tablet replica
I20260808 11:40:55.632176 14956 raft_consensus.cc:2229] T 366fafae0c1d4f8b8677f64b062e9c2e P edff6837b14e4df1afe5b23c274ab44e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:40:55.632349 14953 raft_consensus.cc:2229] T 366fafae0c1d4f8b8677f64b062e9c2e P 3d02a3b7b1384997b48d3255f708864c [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:40:55.632896 14954 raft_consensus.cc:2258] T 366fafae0c1d4f8b8677f64b062e9c2e P d77df0c06c4b49288f5d8feddb6e9691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:40:55.633096 14953 raft_consensus.cc:2258] T 366fafae0c1d4f8b8677f64b062e9c2e P 3d02a3b7b1384997b48d3255f708864c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:40:55.633481 14956 raft_consensus.cc:2258] T 366fafae0c1d4f8b8677f64b062e9c2e P edff6837b14e4df1afe5b23c274ab44e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:40:55.635782 14956 ts_tablet_manager.cc:1905] T 366fafae0c1d4f8b8677f64b062e9c2e P edff6837b14e4df1afe5b23c274ab44e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:40:55.635788 14953 ts_tablet_manager.cc:1905] T 366fafae0c1d4f8b8677f64b062e9c2e P 3d02a3b7b1384997b48d3255f708864c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:40:55.636992 14954 ts_tablet_manager.cc:1905] T 366fafae0c1d4f8b8677f64b062e9c2e P d77df0c06c4b49288f5d8feddb6e9691: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 08 11:40:55 dist-test-slave-tc2s krb5kdc[14111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.65@KRBTEST.COM
I20260808 11:40:55.643492 14953 ts_tablet_manager.cc:1918] T 366fafae0c1d4f8b8677f64b062e9c2e P 3d02a3b7b1384997b48d3255f708864c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:40:55.643699 14953 log.cc:1196] T 366fafae0c1d4f8b8677f64b062e9c2e P 3d02a3b7b1384997b48d3255f708864c: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/wal/wals/366fafae0c1d4f8b8677f64b062e9c2e
I20260808 11:40:55.644273 14953 ts_tablet_manager.cc:1939] T 366fafae0c1d4f8b8677f64b062e9c2e P 3d02a3b7b1384997b48d3255f708864c: Deleting consensus metadata
I20260808 11:40:55.646272 14155 catalog_manager.cc:4841] TS 3d02a3b7b1384997b48d3255f708864c (127.24.76.67:35923): tablet 366fafae0c1d4f8b8677f64b062e9c2e (table test-table [id=3be7658c1c6f413fbd0bb6e900dbd9f1]) successfully deleted
I20260808 11:40:55.647852 14954 ts_tablet_manager.cc:1918] T 366fafae0c1d4f8b8677f64b062e9c2e P d77df0c06c4b49288f5d8feddb6e9691: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:40:55.648049 14954 log.cc:1196] T 366fafae0c1d4f8b8677f64b062e9c2e P d77df0c06c4b49288f5d8feddb6e9691: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/wal/wals/366fafae0c1d4f8b8677f64b062e9c2e
I20260808 11:40:55.648550 14956 ts_tablet_manager.cc:1918] T 366fafae0c1d4f8b8677f64b062e9c2e P edff6837b14e4df1afe5b23c274ab44e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:40:55.648612 14954 ts_tablet_manager.cc:1939] T 366fafae0c1d4f8b8677f64b062e9c2e P d77df0c06c4b49288f5d8feddb6e9691: Deleting consensus metadata
I20260808 11:40:55.648752 14956 log.cc:1196] T 366fafae0c1d4f8b8677f64b062e9c2e P edff6837b14e4df1afe5b23c274ab44e: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/wal/wals/366fafae0c1d4f8b8677f64b062e9c2e
I20260808 11:40:55.649329 14956 ts_tablet_manager.cc:1939] T 366fafae0c1d4f8b8677f64b062e9c2e P edff6837b14e4df1afe5b23c274ab44e: Deleting consensus metadata
I20260808 11:40:55.650311 14159 catalog_manager.cc:4841] TS d77df0c06c4b49288f5d8feddb6e9691 (127.24.76.65:42507): tablet 366fafae0c1d4f8b8677f64b062e9c2e (table test-table [id=3be7658c1c6f413fbd0bb6e900dbd9f1]) successfully deleted
I20260808 11:40:55.651010 14154 catalog_manager.cc:4841] TS edff6837b14e4df1afe5b23c274ab44e (127.24.76.66:43777): tablet 366fafae0c1d4f8b8677f64b062e9c2e (table test-table [id=3be7658c1c6f413fbd0bb6e900dbd9f1]) successfully deleted
I20260808 11:40:55.651751 14910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:55.620378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49230 (local address 127.24.76.65:42507)
0808 11:40:55.620659 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:55.620666 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:55.631317 (+ 10651us) server_negotiation.cc:407] Connection header received
0808 11:40:55.631489 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:55.631494 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:55.631573 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:55.631683 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:55.633872 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:55.635010 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:55.636931 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:55.637560 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:55.637657 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:55.643369 (+ 5712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:55.643384 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:55.643401 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:55.643441 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:55.648351 (+ 4910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:55.649381 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:55.649391 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:55.649394 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:55.649576 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:55.650102 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:55.650107 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:55.650110 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:55.650632 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:55.650944 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:55.651265 (+ 321us) server_negotiation.cc:299] Negotiation successful
0808 11:40:55.651449 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260808 11:40:55.652438 14466 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:40:55.772976 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 14266
I20260808 11:40:55.808275 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 14491
Aug 08 11:40:55 dist-test-slave-tc2s krb5kdc[14111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.66@KRBTEST.COM
I20260808 11:40:55.825728 14917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:55.809080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60768 (local address 127.24.76.66:43777)
0808 11:40:55.809259 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:55.809263 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:55.809281 (+ 18us) server_negotiation.cc:407] Connection header received
0808 11:40:55.809472 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:55.809477 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:55.809550 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:55.809635 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:55.811204 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:55.811927 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:55.813783 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:55.814250 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:55.814343 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:55.819393 (+ 5050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:55.819412 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:55.819434 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:55.819485 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:55.823049 (+ 3564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:55.823682 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:55.823692 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:55.823695 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:55.823844 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:55.824140 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:55.824145 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:55.824148 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:55.824631 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:55.824929 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:55.825118 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 11:40:55.825345 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 11:40:55.826473 14670 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:40:55.913997 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 14491
I20260808 11:40:55.945230 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 14684
Aug 08 11:40:55 dist-test-slave-tc2s krb5kdc[14111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.67@KRBTEST.COM
I20260808 11:40:55.960851 14908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:55.946118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60754 (local address 127.24.76.67:35923)
0808 11:40:55.946322 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:55.946328 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:55.946352 (+ 24us) server_negotiation.cc:407] Connection header received
0808 11:40:55.946503 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:55.946507 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:55.946573 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:55.946674 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:55.947929 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:55.948741 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:55.950207 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:55.950693 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:55.950814 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:55.955042 (+ 4228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:55.955054 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:55.955070 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:55.955106 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:55.958071 (+ 2965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:55.958707 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:55.958715 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:55.958717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:55.958871 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:55.959264 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:55.959268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:55.959270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:55.959655 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:55.959948 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:55.960273 (+ 325us) server_negotiation.cc:299] Negotiation successful
0808 11:40:55.960505 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 11:40:55.961524 14876 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:40:56.082957 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 14684
I20260808 11:40:56.109570 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 14128
I20260808 11:40:56.110683 14224 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:40:56.218967 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 14128
I20260808 11:40:56.247990 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:36973
--webserver_interface=127.24.76.126
--webserver_port=33745
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:34219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:36973
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:40:56.575668 14964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:40:56.575989 14964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:40:56.576126 14964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:40:56.585307 14964 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:40:56.585475 14964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:40:56.585589 14964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:40:56.585695 14964 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:40:56.585780 14964 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:40:56.598695 14964 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:34219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:36973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:36973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=33745
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:40:56.600404 14964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:40:56.602367 14964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:40:56.632666 14978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:56.632732 14976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:56.632697 14975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:40:56.634473 14964 server_base.cc:1034] running on GCE node
I20260808 11:40:56.635347 14964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:40:56.636935 14964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:40:56.638136 14964 hybrid_clock.cc:648] HybridClock initialized: now 1786189256638082 us; error 43 us; skew 500 ppm
I20260808 11:40:56.638530 14964 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:40:56 dist-test-slave-tc2s krb5kdc[14111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189256, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:40:56.642170 14964 init.cc:377] Logged in from keytab as kudu/127.24.76.126@KRBTEST.COM (short username kudu)
I20260808 11:40:56.643854 14964 webserver.cc:460] Webserver started at http://127.24.76.126:33745/ using document root <none> and password file <none>
I20260808 11:40:56.644610 14964 fs_manager.cc:359] Metadata directory not provided
I20260808 11:40:56.644730 14964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:40:56.650667 14964 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 11:40:56.653474 14989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:56.654767 14964 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 11:40:56.654935 14964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "458b6770d36f44afb76ef710569715d2"
format_stamp: "Formatted at 2026-08-08 11:40:53 on dist-test-slave-tc2s"
server_key: "3dcafd6cb11bcf112adf724879de3f73"
server_key_iv: "d413c28bccec335c3d166ef34022840d"
server_key_version: "encryptionkey@0"
I20260808 11:40:56.655974 14964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:40:56.668843 14964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:40:56.672955 14964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:40:56.673213 14964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:40:56.690850 14964 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:36973
I20260808 11:40:56.690867 15069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:36973 every 8 connection(s)
I20260808 11:40:56.692390 14964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:40:56.697526 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 14964
I20260808 11:40:56.698375 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.65:42507
--local_ip_for_outbound_sockets=127.24.76.65
--tserver_master_addrs=127.24.76.126:36973
--webserver_port=44759
--webserver_interface=127.24.76.65
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:34219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:40:56.699250 15070 sys_catalog.cc:263] Verifying existing consensus state
I20260808 11:40:56.702450 15070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2: Bootstrap starting.
I20260808 11:40:56.721352 15070 log.cc:824] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2: Log is configured to *not* fsync() on all Append() calls
I20260808 11:40:56.733238 15070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 11:40:56.733700 15070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2: Bootstrap complete.
I20260808 11:40:56.742760 15070 raft_consensus.cc:348] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "458b6770d36f44afb76ef710569715d2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 36973 } }
I20260808 11:40:56.743564 15070 raft_consensus.cc:729] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 458b6770d36f44afb76ef710569715d2, State: Initialized, Role: FOLLOWER
I20260808 11:40:56.744113 15070 consensus_queue.cc:260] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "458b6770d36f44afb76ef710569715d2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 36973 } }
I20260808 11:40:56.744349 15070 raft_consensus.cc:388] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:40:56.744458 15070 raft_consensus.cc:482] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:40:56.744609 15070 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [term 1 FOLLOWER]: Advancing to term 2
I20260808 11:40:56.748296 15070 raft_consensus.cc:504] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "458b6770d36f44afb76ef710569715d2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 36973 } }
I20260808 11:40:56.748786 15070 leader_election.cc:302] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 458b6770d36f44afb76ef710569715d2; no voters:
I20260808 11:40:56.749364 15070 leader_election.cc:288] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 11:40:56.749534 15076 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 11:40:56.750661 15076 raft_consensus.cc:686] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [term 2 LEADER]: Becoming Leader. State: Replica: 458b6770d36f44afb76ef710569715d2, State: Running, Role: LEADER
I20260808 11:40:56.751277 15076 consensus_queue.cc:237] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "458b6770d36f44afb76ef710569715d2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 36973 } }
I20260808 11:40:56.751616 15070 sys_catalog.cc:564] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:40:56.753773 15079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 458b6770d36f44afb76ef710569715d2. Latest consensus state: current_term: 2 leader_uuid: "458b6770d36f44afb76ef710569715d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "458b6770d36f44afb76ef710569715d2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 36973 } } }
I20260808 11:40:56.754195 15079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [sys.catalog]: This master's current role is: LEADER
I20260808 11:40:56.754792 15077 sys_catalog.cc:455] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "458b6770d36f44afb76ef710569715d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "458b6770d36f44afb76ef710569715d2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 36973 } } }
I20260808 11:40:56.755095 15077 sys_catalog.cc:458] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [sys.catalog]: This master's current role is: LEADER
I20260808 11:40:56.758837 15082 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:40:56.772500 15082 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:40:56.774874 15082 catalog_manager.cc:1259] Loaded cluster ID: 56e1e86b3cc14593a8d52c78152b399d
I20260808 11:40:56.775068 15082 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:40:56.779132 15082 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 11:40:56.789155 15102 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 11:40:56.794586 15082 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 11:40:56.801195 15082 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:40:56.802475 15082 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2: Loaded TSK: 0
I20260808 11:40:56.803498 15082 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 11:40:57.054493 15073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:40:57.054759 15073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:40:57.054888 15073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:40:57.062371 15073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:40:57.062479 15073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:40:57.062636 15073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.65
I20260808 11:40:57.071981 15073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:34219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.65:42507
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.76.65
--webserver_port=44759
--tserver_master_addrs=127.24.76.126:36973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.65
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:40:57.073181 15073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:40:57.074479 15073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:40:57.089123 15112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:57.089108 15113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:57.089095 15122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:40:57.089764 15073 server_base.cc:1034] running on GCE node
I20260808 11:40:57.090523 15073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:40:57.091650 15073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:40:57.092893 15073 hybrid_clock.cc:648] HybridClock initialized: now 1786189257092830 us; error 47 us; skew 500 ppm
I20260808 11:40:57.093241 15073 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:40:57 dist-test-slave-tc2s krb5kdc[14111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189257, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:40:57.096747 15073 init.cc:377] Logged in from keytab as kudu/127.24.76.65@KRBTEST.COM (short username kudu)
I20260808 11:40:57.098172 15073 webserver.cc:460] Webserver started at http://127.24.76.65:44759/ using document root <none> and password file <none>
I20260808 11:40:57.098814 15073 fs_manager.cc:359] Metadata directory not provided
I20260808 11:40:57.098950 15073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:40:57.104130 15073 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.005s
I20260808 11:40:57.106849 15131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:57.108172 15073 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 11:40:57.108322 15073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/wal
uuid: "d77df0c06c4b49288f5d8feddb6e9691"
format_stamp: "Formatted at 2026-08-08 11:40:54 on dist-test-slave-tc2s"
server_key: "1dd4a2a594b34009141005e2ea168904"
server_key_iv: "de79f555936eb8ac8c0d68a0a070a8eb"
server_key_version: "encryptionkey@0"
I20260808 11:40:57.109249 15073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:40:57.124773 15073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:40:57.128703 15073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:40:57.128934 15073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:40:57.129850 15073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:40:57.131196 15073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:40:57.131300 15073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:57.131400 15073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:40:57.131474 15073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:57.161211 15073 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.65:42507
I20260808 11:40:57.161283 15302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.65:42507 every 8 connection(s)
I20260808 11:40:57.162660 15073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
I20260808 11:40:57.166262 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 15073
I20260808 11:40:57.166903 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.66:43777
--local_ip_for_outbound_sockets=127.24.76.66
--tserver_master_addrs=127.24.76.126:36973
--webserver_port=34609
--webserver_interface=127.24.76.66
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:34219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 11:40:57 dist-test-slave-tc2s krb5kdc[14111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189257, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:40:57.184930 15306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:57.165224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:57877 (local address 127.24.76.126:36973)
0808 11:40:57.165815 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:57.165831 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:57.166340 (+ 509us) server_negotiation.cc:407] Connection header received
0808 11:40:57.168411 (+ 2071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:57.168423 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:57.168726 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:57.169043 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:57.170591 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:57.171806 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:57.173282 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:57.173940 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:57.174132 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:57.177707 (+ 3575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:57.177717 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:57.177734 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:57.177778 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:57.180888 (+ 3110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:57.181675 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:57.181681 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:57.181687 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:57.181847 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:57.182298 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:57.182301 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:57.182304 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:57.182742 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:57.182876 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:57.183692 (+ 816us) server_negotiation.cc:299] Negotiation successful
0808 11:40:57.184000 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":226,"threads_started":1}
I20260808 11:40:57.185461 15307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:57.165393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.65:57877)
0808 11:40:57.166171 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:57.166215 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:57.167909 (+ 1694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:57.169219 (+ 1310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:57.169229 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:57.169796 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:40:57.170374 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:57.170392 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:57.171957 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:57.171961 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:57.173142 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:57.173155 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:57.174167 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:57.174175 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:57.174437 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:57.175038 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:40:57.175060 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:40:57.177390 (+ 2330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:40:57.181039 (+ 3649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:57.181049 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:57.181066 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:57.181571 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:57.181976 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:57.181980 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:57.181983 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:57.182205 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:57.182914 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:40:57.182921 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:40:57.183274 (+ 353us) client_negotiation.cc:769] Sending connection context
0808 11:40:57.183537 (+ 263us) client_negotiation.cc:240] Negotiation successful
0808 11:40:57.184027 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":317,"threads_started":1}
I20260808 11:40:57.188295 15303 heartbeater.cc:347] Connected to a master server at 127.24.76.126:36973
I20260808 11:40:57.188563 15303 heartbeater.cc:464] Registering TS with master...
I20260808 11:40:57.189267 15303 heartbeater.cc:511] Master 127.24.76.126:36973 requested a full tablet report, sending...
I20260808 11:40:57.191358 15022 ts_manager.cc:194] Registered new tserver with Master: d77df0c06c4b49288f5d8feddb6e9691 (127.24.76.65:42507)
I20260808 11:40:57.192795 15022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.65@KRBTEST.COM'} at 127.24.76.65:57877
W20260808 11:40:57.498863 15308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:40:57.499115 15308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:40:57.499248 15308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:40:57.506357 15308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:40:57.506464 15308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:40:57.506620 15308 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.66
I20260808 11:40:57.515898 15308 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:34219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.66:43777
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.76.66
--webserver_port=34609
--tserver_master_addrs=127.24.76.126:36973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.66
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:40:57.517055 15308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:40:57.518358 15308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:40:57.530694 15316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:57.530781 15315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:57.530810 15318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:40:57.531697 15308 server_base.cc:1034] running on GCE node
I20260808 11:40:57.532305 15308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:40:57.533226 15308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:40:57.534456 15308 hybrid_clock.cc:648] HybridClock initialized: now 1786189257534395 us; error 48 us; skew 500 ppm
I20260808 11:40:57.534758 15308 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:40:57 dist-test-slave-tc2s krb5kdc[14111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189257, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:40:57.537835 15308 init.cc:377] Logged in from keytab as kudu/127.24.76.66@KRBTEST.COM (short username kudu)
I20260808 11:40:57.539213 15308 webserver.cc:460] Webserver started at http://127.24.76.66:34609/ using document root <none> and password file <none>
I20260808 11:40:57.539860 15308 fs_manager.cc:359] Metadata directory not provided
I20260808 11:40:57.539985 15308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:40:57.545125 15308 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.001s
I20260808 11:40:57.547817 15330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:57.549103 15308 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 11:40:57.549249 15308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/wal
uuid: "edff6837b14e4df1afe5b23c274ab44e"
format_stamp: "Formatted at 2026-08-08 11:40:54 on dist-test-slave-tc2s"
server_key: "0c56724677c9398dc750b305ffa0b3c7"
server_key_iv: "722b13ef4d67a0ef14f375f71f1c74b8"
server_key_version: "encryptionkey@0"
I20260808 11:40:57.550233 15308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:40:57.565094 15308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:40:57.569031 15308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:40:57.569273 15308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:40:57.570192 15308 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:40:57.571468 15308 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:40:57.571555 15308 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:57.571659 15308 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:40:57.571731 15308 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:57.600656 15308 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.66:43777
I20260808 11:40:57.600720 15502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.66:43777 every 8 connection(s)
I20260808 11:40:57.602041 15308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
I20260808 11:40:57.606853 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 15308
I20260808 11:40:57.607431 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.67:35923
--local_ip_for_outbound_sockets=127.24.76.67
--tserver_master_addrs=127.24.76.126:36973
--webserver_port=34563
--webserver_interface=127.24.76.67
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:34219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 11:40:57 dist-test-slave-tc2s krb5kdc[14111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189257, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:40:57.621300 15306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:57.604407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:42713 (local address 127.24.76.126:36973)
0808 11:40:57.604599 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:57.604603 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:57.605686 (+ 1083us) server_negotiation.cc:407] Connection header received
0808 11:40:57.607566 (+ 1880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:57.607572 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:57.607662 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:57.607763 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:57.609188 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:57.610142 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:57.611625 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:57.612238 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:57.612352 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:57.615698 (+ 3346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:57.615704 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:57.615708 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:57.615746 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:57.618510 (+ 2764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:57.619191 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:57.619196 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:57.619199 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:57.619357 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:57.619729 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:57.619733 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:57.619735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:57.620073 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:57.620229 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:57.620888 (+ 659us) server_negotiation.cc:299] Negotiation successful
0808 11:40:57.621064 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 11:40:57.622532 15505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:57.604843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.66:42713)
0808 11:40:57.605532 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:57.605568 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:57.607305 (+ 1737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:57.607955 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:57.607963 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:57.608423 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:40:57.609014 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:57.609033 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:57.610278 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:57.610281 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:57.611464 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:57.611476 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:57.612421 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:57.612426 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:57.612638 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:57.613247 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:40:57.613276 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:40:57.615489 (+ 2213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:40:57.618647 (+ 3158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:57.618653 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:57.618663 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:57.619076 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:57.619449 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:57.619451 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:57.619454 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:57.619634 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:57.620176 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:40:57.620181 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:40:57.620526 (+ 345us) client_negotiation.cc:769] Sending connection context
0808 11:40:57.620788 (+ 262us) client_negotiation.cc:240] Negotiation successful
0808 11:40:57.621276 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":263,"threads_started":1}
I20260808 11:40:57.623903 15503 heartbeater.cc:347] Connected to a master server at 127.24.76.126:36973
I20260808 11:40:57.624246 15503 heartbeater.cc:464] Registering TS with master...
I20260808 11:40:57.625079 15503 heartbeater.cc:511] Master 127.24.76.126:36973 requested a full tablet report, sending...
I20260808 11:40:57.626521 15022 ts_manager.cc:194] Registered new tserver with Master: edff6837b14e4df1afe5b23c274ab44e (127.24.76.66:43777)
I20260808 11:40:57.627307 15022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.66@KRBTEST.COM'} at 127.24.76.66:42713
W20260808 11:40:57.937773 15506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:40:57.938068 15506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:40:57.938205 15506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:40:57.945056 15506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:40:57.945158 15506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:40:57.945309 15506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.67
I20260808 11:40:57.954631 15506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:34219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.67:35923
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.76.67
--webserver_port=34563
--tserver_master_addrs=127.24.76.126:36973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.67
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:40:57.955747 15506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:40:57.956998 15506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:40:57.969390 15514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:57.969453 15515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:57.969610 15518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:40:57.970523 15506 server_base.cc:1034] running on GCE node
I20260808 11:40:57.971056 15506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:40:57.971894 15506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:40:57.973088 15506 hybrid_clock.cc:648] HybridClock initialized: now 1786189257973042 us; error 40 us; skew 500 ppm
I20260808 11:40:57.973364 15506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:40:57 dist-test-slave-tc2s krb5kdc[14111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189257, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:40:57.976267 15506 init.cc:377] Logged in from keytab as kudu/127.24.76.67@KRBTEST.COM (short username kudu)
I20260808 11:40:57.977628 15506 webserver.cc:460] Webserver started at http://127.24.76.67:34563/ using document root <none> and password file <none>
I20260808 11:40:57.978319 15506 fs_manager.cc:359] Metadata directory not provided
I20260808 11:40:57.978469 15506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:40:57.983477 15506 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 11:40:57.985985 15527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:57.987226 15506 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:40:57.987375 15506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/wal
uuid: "3d02a3b7b1384997b48d3255f708864c"
format_stamp: "Formatted at 2026-08-08 11:40:55 on dist-test-slave-tc2s"
server_key: "9e705ee2ee6732036a0f3465c60d9873"
server_key_iv: "611ad4bceec41c52e7f0cdd1e38a1bc0"
server_key_version: "encryptionkey@0"
I20260808 11:40:57.988291 15506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:40:57.999783 15506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:40:58.004814 15506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:40:58.005105 15506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:40:58.007643 15506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:40:58.009311 15506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:40:58.009399 15506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:58.009477 15506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:40:58.009528 15506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:58.038964 15506 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.67:35923
I20260808 11:40:58.039040 15683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.67:35923 every 8 connection(s)
I20260808 11:40:58.040297 15506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
I20260808 11:40:58.048122 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 15506
Aug 08 11:40:58 dist-test-slave-tc2s krb5kdc[14111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189257, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:40:58.059401 15306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:58.042632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:40629 (local address 127.24.76.126:36973)
0808 11:40:58.042815 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:58.042820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:58.043882 (+ 1062us) server_negotiation.cc:407] Connection header received
0808 11:40:58.045568 (+ 1686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:58.045573 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:58.045638 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:58.045721 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:58.047145 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:58.047840 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:58.049585 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:58.050162 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:58.050261 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:58.053696 (+ 3435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:58.053701 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:58.053704 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:58.053733 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:58.056494 (+ 2761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:58.057153 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:58.057158 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:58.057162 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:58.057285 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:58.057687 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:58.057692 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:58.057696 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:58.057991 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:58.058121 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:58.058761 (+ 640us) server_negotiation.cc:299] Negotiation successful
0808 11:40:58.059009 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 11:40:58.060467 15688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:58.043057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.67:40629)
0808 11:40:58.043722 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:58.043765 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:58.045330 (+ 1565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:58.045944 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:58.045953 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:58.046433 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:40:58.046980 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:58.046999 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:58.047992 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:58.047997 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:58.049399 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:58.049415 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:58.050373 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:58.050381 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:58.050637 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:58.051227 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:40:58.051247 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:40:58.053521 (+ 2274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:40:58.056620 (+ 3099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:58.056628 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:58.056643 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:58.057044 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:58.057388 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:40:58.057393 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:40:58.057397 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:40:58.057581 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:40:58.058120 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:40:58.058127 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:40:58.058411 (+ 284us) client_negotiation.cc:769] Sending connection context
0808 11:40:58.058645 (+ 234us) client_negotiation.cc:240] Negotiation successful
0808 11:40:58.059126 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":250,"threads_started":1}
I20260808 11:40:58.061842 15685 heartbeater.cc:347] Connected to a master server at 127.24.76.126:36973
I20260808 11:40:58.062187 15685 heartbeater.cc:464] Registering TS with master...
I20260808 11:40:58.063001 15685 heartbeater.cc:511] Master 127.24.76.126:36973 requested a full tablet report, sending...
I20260808 11:40:58.063663 15690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:58.048972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48230 (local address 127.24.76.126:36973)
0808 11:40:58.049468 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:58.049473 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:58.050008 (+ 535us) server_negotiation.cc:407] Connection header received
0808 11:40:58.050171 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:58.050175 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:58.050236 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:58.050350 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:58.051818 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:58.052749 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:58.055088 (+ 2339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:58.055642 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:58.055759 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:58.058214 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:58.058219 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:58.058222 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:58.058249 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:58.061182 (+ 2933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:58.061868 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:58.061872 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:58.061874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:58.062058 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:58.062472 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:58.062476 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:58.062478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:58.062719 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:58.062892 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:58.063265 (+ 373us) server_negotiation.cc:299] Negotiation successful
0808 11:40:58.063412 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":231,"threads_started":1}
I20260808 11:40:58.064491 15022 ts_manager.cc:194] Registered new tserver with Master: 3d02a3b7b1384997b48d3255f708864c (127.24.76.67:35923)
I20260808 11:40:58.065351 15022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.67@KRBTEST.COM'} at 127.24.76.67:40629
I20260808 11:40:58.066048 24881 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:40:58.072064 15690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:58.066977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48236 (local address 127.24.76.126:36973)
0808 11:40:58.067164 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:58.067170 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:58.067598 (+ 428us) server_negotiation.cc:407] Connection header received
0808 11:40:58.067770 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:58.067774 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:58.067883 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:58.067996 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:40:58.068353 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:58.068991 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:58.070379 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:58.070783 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:58.070868 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:58.071243 (+ 375us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:40:58.071623 (+ 380us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:40:58.071718 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:58.071785 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 11:40:58.071839 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 11:40:58.072824 15018 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:48236:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:40:58.079942 15018 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:40:58.102895 15709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:58.091352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49234 (local address 127.24.76.65:42507)
0808 11:40:58.092623 (+ 1271us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:58.092630 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:58.093202 (+ 572us) server_negotiation.cc:407] Connection header received
0808 11:40:58.093490 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:58.093500 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:58.093734 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:58.093931 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:40:58.094925 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:58.096582 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:58.100134 (+ 3552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:58.101652 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:58.101791 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:58.102015 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:58.102388 (+ 373us) server_negotiation.cc:299] Negotiation successful
0808 11:40:58.102556 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1120,"thread_start_us":1140,"threads_started":1}
I20260808 11:40:58.103562 15705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:58.090573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:42507 (local address 127.0.0.1:49234)
0808 11:40:58.093084 (+ 2511us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:58.093106 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:58.093305 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:58.094273 (+ 968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:58.094281 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:58.094311 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:40:58.094717 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:58.094729 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:58.096804 (+ 2075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:58.096810 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:58.099301 (+ 2491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:58.099318 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:58.101865 (+ 2547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:58.101872 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:58.102102 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:58.102160 (+ 58us) client_negotiation.cc:769] Sending connection context
0808 11:40:58.103152 (+ 992us) client_negotiation.cc:240] Negotiation successful
0808 11:40:58.103308 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2384,"thread_start_us":2244,"threads_started":1}
I20260808 11:40:58.107271 15196 tablet_service.cc:1467] Processing CreateTablet for tablet 30b4d9cc324f47d28c5d84bf03e91390 (DEFAULT_TABLE table=test-table [id=2c62ff700f0d4ef0a5885f4715449fec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:40:58.108465 15708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:58.090547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:43777 (local address 127.0.0.1:60778)
0808 11:40:58.091661 (+ 1114us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:58.091711 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:58.091864 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:58.102309 (+ 10445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:58.102314 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:58.102345 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:40:58.102821 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:58.102834 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:58.104595 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:58.104600 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:58.106031 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:58.106046 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:58.107623 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:58.107629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:58.107853 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:58.107887 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 11:40:58.107971 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 11:40:58.108111 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":855,"mutex_wait_us":36,"thread_start_us":9044,"threads_started":1}
I20260808 11:40:58.108548 15706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:58.090691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60778 (local address 127.24.76.66:43777)
0808 11:40:58.101693 (+ 11002us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:58.101701 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:58.101732 (+ 31us) server_negotiation.cc:407] Connection header received
0808 11:40:58.101830 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:58.101837 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:58.102073 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:58.102309 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:40:58.103024 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:58.104452 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:58.106197 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:58.107381 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:58.107769 (+ 388us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:58.107969 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:58.108044 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 11:40:58.108195 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10846,"thread_start_us":10723,"threads_started":1}
I20260808 11:40:58.108733 15196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30b4d9cc324f47d28c5d84bf03e91390. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:40:58.111753 15704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:58.090921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:35923 (local address 127.0.0.1:60762)
0808 11:40:58.099855 (+ 8934us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:58.099872 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:58.099990 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:58.102114 (+ 2124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:58.102119 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:58.102142 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:40:58.102624 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:58.102635 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:58.104598 (+ 1963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:58.104602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:58.109194 (+ 4592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:58.109209 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:58.110970 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:58.110976 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:58.111141 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:58.111173 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 11:40:58.111261 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 11:40:58.111376 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8814,"mutex_wait_us":55,"thread_start_us":663,"threads_started":1}
I20260808 11:40:58.111814 15710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:58.092366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60762 (local address 127.24.76.67:35923)
0808 11:40:58.101386 (+ 9020us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:58.101395 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:58.101429 (+ 34us) server_negotiation.cc:407] Connection header received
0808 11:40:58.101528 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:58.101535 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:58.101745 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:58.101961 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:40:58.102792 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:58.104430 (+ 1638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:58.109381 (+ 4951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:58.110755 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:58.110882 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:58.111095 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:58.111344 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 11:40:58.111515 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8842,"thread_start_us":8664,"threads_started":1}
I20260808 11:40:58.111577 15410 tablet_service.cc:1467] Processing CreateTablet for tablet 30b4d9cc324f47d28c5d84bf03e91390 (DEFAULT_TABLE table=test-table [id=2c62ff700f0d4ef0a5885f4715449fec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:40:58.112908 15410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30b4d9cc324f47d28c5d84bf03e91390. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:40:58.115177 15597 tablet_service.cc:1467] Processing CreateTablet for tablet 30b4d9cc324f47d28c5d84bf03e91390 (DEFAULT_TABLE table=test-table [id=2c62ff700f0d4ef0a5885f4715449fec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:40:58.116430 15597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30b4d9cc324f47d28c5d84bf03e91390. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:40:58.121531 15711 tablet_bootstrap.cc:492] T 30b4d9cc324f47d28c5d84bf03e91390 P d77df0c06c4b49288f5d8feddb6e9691: Bootstrap starting.
I20260808 11:40:58.124604 15711 tablet_bootstrap.cc:654] T 30b4d9cc324f47d28c5d84bf03e91390 P d77df0c06c4b49288f5d8feddb6e9691: Neither blocks nor log segments found. Creating new log.
I20260808 11:40:58.125550 15711 log.cc:824] T 30b4d9cc324f47d28c5d84bf03e91390 P d77df0c06c4b49288f5d8feddb6e9691: Log is configured to *not* fsync() on all Append() calls
I20260808 11:40:58.128813 15711 tablet_bootstrap.cc:492] T 30b4d9cc324f47d28c5d84bf03e91390 P d77df0c06c4b49288f5d8feddb6e9691: No bootstrap required, opened a new log
I20260808 11:40:58.129069 15711 ts_tablet_manager.cc:1397] T 30b4d9cc324f47d28c5d84bf03e91390 P d77df0c06c4b49288f5d8feddb6e9691: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20260808 11:40:58.129216 15713 tablet_bootstrap.cc:492] T 30b4d9cc324f47d28c5d84bf03e91390 P edff6837b14e4df1afe5b23c274ab44e: Bootstrap starting.
I20260808 11:40:58.130203 15714 tablet_bootstrap.cc:492] T 30b4d9cc324f47d28c5d84bf03e91390 P 3d02a3b7b1384997b48d3255f708864c: Bootstrap starting.
I20260808 11:40:58.132280 15713 tablet_bootstrap.cc:654] T 30b4d9cc324f47d28c5d84bf03e91390 P edff6837b14e4df1afe5b23c274ab44e: Neither blocks nor log segments found. Creating new log.
I20260808 11:40:58.132812 15714 tablet_bootstrap.cc:654] T 30b4d9cc324f47d28c5d84bf03e91390 P 3d02a3b7b1384997b48d3255f708864c: Neither blocks nor log segments found. Creating new log.
I20260808 11:40:58.133441 15713 log.cc:824] T 30b4d9cc324f47d28c5d84bf03e91390 P edff6837b14e4df1afe5b23c274ab44e: Log is configured to *not* fsync() on all Append() calls
I20260808 11:40:58.133714 15714 log.cc:824] T 30b4d9cc324f47d28c5d84bf03e91390 P 3d02a3b7b1384997b48d3255f708864c: Log is configured to *not* fsync() on all Append() calls
I20260808 11:40:58.136152 15713 tablet_bootstrap.cc:492] T 30b4d9cc324f47d28c5d84bf03e91390 P edff6837b14e4df1afe5b23c274ab44e: No bootstrap required, opened a new log
I20260808 11:40:58.136366 15713 ts_tablet_manager.cc:1397] T 30b4d9cc324f47d28c5d84bf03e91390 P edff6837b14e4df1afe5b23c274ab44e: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20260808 11:40:58.137019 15714 tablet_bootstrap.cc:492] T 30b4d9cc324f47d28c5d84bf03e91390 P 3d02a3b7b1384997b48d3255f708864c: No bootstrap required, opened a new log
I20260808 11:40:58.137269 15714 ts_tablet_manager.cc:1397] T 30b4d9cc324f47d28c5d84bf03e91390 P 3d02a3b7b1384997b48d3255f708864c: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260808 11:40:58.138777 15711 raft_consensus.cc:348] T 30b4d9cc324f47d28c5d84bf03e91390 P d77df0c06c4b49288f5d8feddb6e9691 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edff6837b14e4df1afe5b23c274ab44e" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 43777 } } peers { permanent_uuid: "d77df0c06c4b49288f5d8feddb6e9691" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 42507 } } peers { permanent_uuid: "3d02a3b7b1384997b48d3255f708864c" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 35923 } }
I20260808 11:40:58.139181 15711 raft_consensus.cc:374] T 30b4d9cc324f47d28c5d84bf03e91390 P d77df0c06c4b49288f5d8feddb6e9691 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:40:58.139279 15711 raft_consensus.cc:729] T 30b4d9cc324f47d28c5d84bf03e91390 P d77df0c06c4b49288f5d8feddb6e9691 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d77df0c06c4b49288f5d8feddb6e9691, State: Initialized, Role: FOLLOWER
I20260808 11:40:58.139861 15711 consensus_queue.cc:260] T 30b4d9cc324f47d28c5d84bf03e91390 P d77df0c06c4b49288f5d8feddb6e9691 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edff6837b14e4df1afe5b23c274ab44e" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 43777 } } peers { permanent_uuid: "d77df0c06c4b49288f5d8feddb6e9691" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 42507 } } peers { permanent_uuid: "3d02a3b7b1384997b48d3255f708864c" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 35923 } }
I20260808 11:40:58.140942 15303 heartbeater.cc:503] Master 127.24.76.126:36973 was elected leader, sending a full tablet report...
I20260808 11:40:58.141547 15711 ts_tablet_manager.cc:1428] T 30b4d9cc324f47d28c5d84bf03e91390 P d77df0c06c4b49288f5d8feddb6e9691: Time spent starting tablet: real 0.012s user 0.011s sys 0.001s
I20260808 11:40:58.145331 15714 raft_consensus.cc:348] T 30b4d9cc324f47d28c5d84bf03e91390 P 3d02a3b7b1384997b48d3255f708864c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edff6837b14e4df1afe5b23c274ab44e" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 43777 } } peers { permanent_uuid: "d77df0c06c4b49288f5d8feddb6e9691" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 42507 } } peers { permanent_uuid: "3d02a3b7b1384997b48d3255f708864c" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 35923 } }
I20260808 11:40:58.145740 15714 raft_consensus.cc:374] T 30b4d9cc324f47d28c5d84bf03e91390 P 3d02a3b7b1384997b48d3255f708864c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:40:58.145833 15714 raft_consensus.cc:729] T 30b4d9cc324f47d28c5d84bf03e91390 P 3d02a3b7b1384997b48d3255f708864c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d02a3b7b1384997b48d3255f708864c, State: Initialized, Role: FOLLOWER
I20260808 11:40:58.145676 15713 raft_consensus.cc:348] T 30b4d9cc324f47d28c5d84bf03e91390 P edff6837b14e4df1afe5b23c274ab44e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edff6837b14e4df1afe5b23c274ab44e" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 43777 } } peers { permanent_uuid: "d77df0c06c4b49288f5d8feddb6e9691" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 42507 } } peers { permanent_uuid: "3d02a3b7b1384997b48d3255f708864c" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 35923 } }
I20260808 11:40:58.146111 15713 raft_consensus.cc:374] T 30b4d9cc324f47d28c5d84bf03e91390 P edff6837b14e4df1afe5b23c274ab44e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:40:58.146207 15713 raft_consensus.cc:729] T 30b4d9cc324f47d28c5d84bf03e91390 P edff6837b14e4df1afe5b23c274ab44e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edff6837b14e4df1afe5b23c274ab44e, State: Initialized, Role: FOLLOWER
I20260808 11:40:58.146525 15714 consensus_queue.cc:260] T 30b4d9cc324f47d28c5d84bf03e91390 P 3d02a3b7b1384997b48d3255f708864c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edff6837b14e4df1afe5b23c274ab44e" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 43777 } } peers { permanent_uuid: "d77df0c06c4b49288f5d8feddb6e9691" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 42507 } } peers { permanent_uuid: "3d02a3b7b1384997b48d3255f708864c" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 35923 } }
I20260808 11:40:58.146834 15713 consensus_queue.cc:260] T 30b4d9cc324f47d28c5d84bf03e91390 P edff6837b14e4df1afe5b23c274ab44e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edff6837b14e4df1afe5b23c274ab44e" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 43777 } } peers { permanent_uuid: "d77df0c06c4b49288f5d8feddb6e9691" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 42507 } } peers { permanent_uuid: "3d02a3b7b1384997b48d3255f708864c" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 35923 } }
I20260808 11:40:58.147610 15685 heartbeater.cc:503] Master 127.24.76.126:36973 was elected leader, sending a full tablet report...
I20260808 11:40:58.147927 15503 heartbeater.cc:503] Master 127.24.76.126:36973 was elected leader, sending a full tablet report...
I20260808 11:40:58.148303 15714 ts_tablet_manager.cc:1428] T 30b4d9cc324f47d28c5d84bf03e91390 P 3d02a3b7b1384997b48d3255f708864c: Time spent starting tablet: real 0.011s user 0.008s sys 0.003s
I20260808 11:40:58.148452 15713 ts_tablet_manager.cc:1428] T 30b4d9cc324f47d28c5d84bf03e91390 P edff6837b14e4df1afe5b23c274ab44e: Time spent starting tablet: real 0.012s user 0.010s sys 0.000s
W20260808 11:40:58.164716 15304 tablet.cc:2369] T 30b4d9cc324f47d28c5d84bf03e91390 P d77df0c06c4b49288f5d8feddb6e9691: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:40:58.291800 15686 tablet.cc:2369] T 30b4d9cc324f47d28c5d84bf03e91390 P 3d02a3b7b1384997b48d3255f708864c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:40:58.345863 15718 raft_consensus.cc:482] T 30b4d9cc324f47d28c5d84bf03e91390 P 3d02a3b7b1384997b48d3255f708864c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:40:58.346196 15718 raft_consensus.cc:504] T 30b4d9cc324f47d28c5d84bf03e91390 P 3d02a3b7b1384997b48d3255f708864c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edff6837b14e4df1afe5b23c274ab44e" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 43777 } } peers { permanent_uuid: "d77df0c06c4b49288f5d8feddb6e9691" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 42507 } } peers { permanent_uuid: "3d02a3b7b1384997b48d3255f708864c" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 35923 } }
I20260808 11:40:58.347914 15718 leader_election.cc:288] T 30b4d9cc324f47d28c5d84bf03e91390 P 3d02a3b7b1384997b48d3255f708864c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers edff6837b14e4df1afe5b23c274ab44e (127.24.76.66:43777), d77df0c06c4b49288f5d8feddb6e9691 (127.24.76.65:42507)
W20260808 11:40:58.353911 15504 tablet.cc:2369] T 30b4d9cc324f47d28c5d84bf03e91390 P edff6837b14e4df1afe5b23c274ab44e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:40:58.354171 15688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:58.348182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:42507 (local address 127.24.76.67:39171)
0808 11:40:58.348478 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:58.348498 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:58.348631 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:58.349051 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:58.349055 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:58.349082 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:40:58.349282 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:58.349294 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:58.350491 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:58.350497 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:58.352073 (+ 1576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:58.352085 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:58.353338 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:58.353344 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:58.353523 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:58.353559 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 11:40:58.353657 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 11:40:58.353844 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148}
I20260808 11:40:58.354216 15709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:58.348348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:39171 (local address 127.24.76.65:42507)
0808 11:40:58.348521 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:58.348526 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:58.348547 (+ 21us) server_negotiation.cc:407] Connection header received
0808 11:40:58.348811 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:58.348817 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:58.348899 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:58.349008 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:40:58.349433 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:58.350363 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:58.352223 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:58.353193 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:58.353319 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:58.353372 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:58.353777 (+ 405us) server_negotiation.cc:299] Negotiation successful
0808 11:40:58.353970 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 11:40:58.354683 15706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:58.348298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:54887 (local address 127.24.76.66:43777)
0808 11:40:58.348508 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:58.348515 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:58.349123 (+ 608us) server_negotiation.cc:407] Connection header received
0808 11:40:58.349290 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:58.349295 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:58.349370 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:58.349458 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:40:58.349996 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:58.350799 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:58.352672 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:58.353593 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:58.353729 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:58.353796 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:58.354320 (+ 524us) server_negotiation.cc:299] Negotiation successful
0808 11:40:58.354460 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 11:40:58.355253 15720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:58.348256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:43777 (local address 127.24.76.67:54887)
0808 11:40:58.349016 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:58.349033 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:40:58.349168 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:40:58.349471 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:40:58.349474 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:40:58.349499 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:40:58.349839 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:58.349850 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:58.350944 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:58.350948 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:58.352520 (+ 1572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:40:58.352535 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:40:58.353948 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:40:58.353955 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:40:58.354126 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:58.354162 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 11:40:58.354831 (+ 669us) client_negotiation.cc:240] Negotiation successful
0808 11:40:58.354957 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":454,"threads_started":1}
I20260808 11:40:58.355240 15234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30b4d9cc324f47d28c5d84bf03e91390" candidate_uuid: "3d02a3b7b1384997b48d3255f708864c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d77df0c06c4b49288f5d8feddb6e9691" is_pre_election: true
I20260808 11:40:58.355792 15234 raft_consensus.cc:2454] T 30b4d9cc324f47d28c5d84bf03e91390 P d77df0c06c4b49288f5d8feddb6e9691 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d02a3b7b1384997b48d3255f708864c in term 0.
I20260808 11:40:58.356067 15441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30b4d9cc324f47d28c5d84bf03e91390" candidate_uuid: "3d02a3b7b1384997b48d3255f708864c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edff6837b14e4df1afe5b23c274ab44e" is_pre_election: true
I20260808 11:40:58.356395 15532 leader_election.cc:302] T 30b4d9cc324f47d28c5d84bf03e91390 P 3d02a3b7b1384997b48d3255f708864c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d02a3b7b1384997b48d3255f708864c, d77df0c06c4b49288f5d8feddb6e9691; no voters:
I20260808 11:40:58.356465 15441 raft_consensus.cc:2454] T 30b4d9cc324f47d28c5d84bf03e91390 P edff6837b14e4df1afe5b23c274ab44e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d02a3b7b1384997b48d3255f708864c in term 0.
I20260808 11:40:58.356748 15718 raft_consensus.cc:2781] T 30b4d9cc324f47d28c5d84bf03e91390 P 3d02a3b7b1384997b48d3255f708864c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:40:58.356897 15718 raft_consensus.cc:482] T 30b4d9cc324f47d28c5d84bf03e91390 P 3d02a3b7b1384997b48d3255f708864c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:40:58.356972 15718 raft_consensus.cc:3037] T 30b4d9cc324f47d28c5d84bf03e91390 P 3d02a3b7b1384997b48d3255f708864c [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:40:58.359141 15718 raft_consensus.cc:504] T 30b4d9cc324f47d28c5d84bf03e91390 P 3d02a3b7b1384997b48d3255f708864c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edff6837b14e4df1afe5b23c274ab44e" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 43777 } } peers { permanent_uuid: "d77df0c06c4b49288f5d8feddb6e9691" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 42507 } } peers { permanent_uuid: "3d02a3b7b1384997b48d3255f708864c" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 35923 } }
I20260808 11:40:58.359764 15718 leader_election.cc:288] T 30b4d9cc324f47d28c5d84bf03e91390 P 3d02a3b7b1384997b48d3255f708864c [CANDIDATE]: Term 1 election: Requested vote from peers edff6837b14e4df1afe5b23c274ab44e (127.24.76.66:43777), d77df0c06c4b49288f5d8feddb6e9691 (127.24.76.65:42507)
I20260808 11:40:58.360157 15441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30b4d9cc324f47d28c5d84bf03e91390" candidate_uuid: "3d02a3b7b1384997b48d3255f708864c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edff6837b14e4df1afe5b23c274ab44e"
I20260808 11:40:58.360253 15234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30b4d9cc324f47d28c5d84bf03e91390" candidate_uuid: "3d02a3b7b1384997b48d3255f708864c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d77df0c06c4b49288f5d8feddb6e9691"
I20260808 11:40:58.360352 15441 raft_consensus.cc:3037] T 30b4d9cc324f47d28c5d84bf03e91390 P edff6837b14e4df1afe5b23c274ab44e [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:40:58.360492 15234 raft_consensus.cc:3037] T 30b4d9cc324f47d28c5d84bf03e91390 P d77df0c06c4b49288f5d8feddb6e9691 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:40:58.362875 15441 raft_consensus.cc:2454] T 30b4d9cc324f47d28c5d84bf03e91390 P edff6837b14e4df1afe5b23c274ab44e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d02a3b7b1384997b48d3255f708864c in term 1.
I20260808 11:40:58.363066 15234 raft_consensus.cc:2454] T 30b4d9cc324f47d28c5d84bf03e91390 P d77df0c06c4b49288f5d8feddb6e9691 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d02a3b7b1384997b48d3255f708864c in term 1.
I20260808 11:40:58.363262 15528 leader_election.cc:302] T 30b4d9cc324f47d28c5d84bf03e91390 P 3d02a3b7b1384997b48d3255f708864c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d02a3b7b1384997b48d3255f708864c, edff6837b14e4df1afe5b23c274ab44e; no voters:
I20260808 11:40:58.363591 15718 raft_consensus.cc:2781] T 30b4d9cc324f47d28c5d84bf03e91390 P 3d02a3b7b1384997b48d3255f708864c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:40:58.364024 15718 raft_consensus.cc:686] T 30b4d9cc324f47d28c5d84bf03e91390 P 3d02a3b7b1384997b48d3255f708864c [term 1 LEADER]: Becoming Leader. State: Replica: 3d02a3b7b1384997b48d3255f708864c, State: Running, Role: LEADER
I20260808 11:40:58.364538 15718 consensus_queue.cc:237] T 30b4d9cc324f47d28c5d84bf03e91390 P 3d02a3b7b1384997b48d3255f708864c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edff6837b14e4df1afe5b23c274ab44e" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 43777 } } peers { permanent_uuid: "d77df0c06c4b49288f5d8feddb6e9691" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 42507 } } peers { permanent_uuid: "3d02a3b7b1384997b48d3255f708864c" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 35923 } }
I20260808 11:40:58.369042 15018 catalog_manager.cc:5495] T 30b4d9cc324f47d28c5d84bf03e91390 P 3d02a3b7b1384997b48d3255f708864c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d02a3b7b1384997b48d3255f708864c (127.24.76.67). New cstate: current_term: 1 leader_uuid: "3d02a3b7b1384997b48d3255f708864c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edff6837b14e4df1afe5b23c274ab44e" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 43777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d77df0c06c4b49288f5d8feddb6e9691" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 42507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d02a3b7b1384997b48d3255f708864c" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 35923 } health_report { overall_health: HEALTHY } } }
I20260808 11:40:58.397842 15710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:58.393065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60768 (local address 127.24.76.67:35923)
0808 11:40:58.393234 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:58.393238 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:58.393293 (+ 55us) server_negotiation.cc:407] Connection header received
0808 11:40:58.393471 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:58.393474 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:58.393558 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:58.393672 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:40:58.393999 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:58.394765 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:58.396133 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:58.396518 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:58.396604 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:58.396993 (+ 389us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:40:58.397446 (+ 453us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:40:58.397551 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:58.397593 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 11:40:58.397654 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 11:40:58.405686 15441 raft_consensus.cc:1261] T 30b4d9cc324f47d28c5d84bf03e91390 P edff6837b14e4df1afe5b23c274ab44e [term 1 FOLLOWER]: Refusing update from remote peer 3d02a3b7b1384997b48d3255f708864c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:40:58.405691 15234 raft_consensus.cc:1261] T 30b4d9cc324f47d28c5d84bf03e91390 P d77df0c06c4b49288f5d8feddb6e9691 [term 1 FOLLOWER]: Refusing update from remote peer 3d02a3b7b1384997b48d3255f708864c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:40:58.406771 15721 consensus_queue.cc:1035] T 30b4d9cc324f47d28c5d84bf03e91390 P 3d02a3b7b1384997b48d3255f708864c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d77df0c06c4b49288f5d8feddb6e9691" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 42507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:40:58.407138 15718 consensus_queue.cc:1035] T 30b4d9cc324f47d28c5d84bf03e91390 P 3d02a3b7b1384997b48d3255f708864c [LEADER]: Connected to new peer: Peer: permanent_uuid: "edff6837b14e4df1afe5b23c274ab44e" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 43777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:40:58.419250 15725 mvcc.cc:205] Tried to move back new op lower bound from 7316231202419363840 to 7316231202264502272. Current Snapshot: MvccSnapshot[applied={T|T < 7316231202419363840}]
I20260808 11:40:58.422081 15728 mvcc.cc:205] Tried to move back new op lower bound from 7316231202419363840 to 7316231202264502272. Current Snapshot: MvccSnapshot[applied={T|T < 7316231202419363840}]
I20260808 11:40:58.432075 15022 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:48236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:40:58.432286 15022 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:48236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:40:58.435416 15022 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2: Sending DeleteTablet for 3 replicas of tablet 30b4d9cc324f47d28c5d84bf03e91390
I20260808 11:40:58.436594 15410 tablet_service.cc:1514] Processing DeleteTablet for tablet 30b4d9cc324f47d28c5d84bf03e91390 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:40:58 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:60778
I20260808 11:40:58.436718 15196 tablet_service.cc:1514] Processing DeleteTablet for tablet 30b4d9cc324f47d28c5d84bf03e91390 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:40:58 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:49234
I20260808 11:40:58.437238 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 15073
I20260808 11:40:58.437574 15597 tablet_service.cc:1514] Processing DeleteTablet for tablet 30b4d9cc324f47d28c5d84bf03e91390 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:40:58 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:60762
I20260808 11:40:58.445571 15739 tablet_replica.cc:330] T 30b4d9cc324f47d28c5d84bf03e91390 P edff6837b14e4df1afe5b23c274ab44e: stopping tablet replica
I20260808 11:40:58.445823 15740 tablet_replica.cc:330] T 30b4d9cc324f47d28c5d84bf03e91390 P d77df0c06c4b49288f5d8feddb6e9691: stopping tablet replica
I20260808 11:40:58.446115 15739 raft_consensus.cc:2229] T 30b4d9cc324f47d28c5d84bf03e91390 P edff6837b14e4df1afe5b23c274ab44e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:40:58.446353 15740 raft_consensus.cc:2229] T 30b4d9cc324f47d28c5d84bf03e91390 P d77df0c06c4b49288f5d8feddb6e9691 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:40:58.446617 15739 raft_consensus.cc:2258] T 30b4d9cc324f47d28c5d84bf03e91390 P edff6837b14e4df1afe5b23c274ab44e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:40:58.446923 15740 raft_consensus.cc:2258] T 30b4d9cc324f47d28c5d84bf03e91390 P d77df0c06c4b49288f5d8feddb6e9691 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:40:58.448062 15741 tablet_replica.cc:330] T 30b4d9cc324f47d28c5d84bf03e91390 P 3d02a3b7b1384997b48d3255f708864c: stopping tablet replica
I20260808 11:40:58.448498 15741 raft_consensus.cc:2229] T 30b4d9cc324f47d28c5d84bf03e91390 P 3d02a3b7b1384997b48d3255f708864c [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:40:58.449260 15741 raft_consensus.cc:2258] T 30b4d9cc324f47d28c5d84bf03e91390 P 3d02a3b7b1384997b48d3255f708864c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:40:58.450277 15740 ts_tablet_manager.cc:1905] T 30b4d9cc324f47d28c5d84bf03e91390 P d77df0c06c4b49288f5d8feddb6e9691: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:40:58.451993 15741 ts_tablet_manager.cc:1905] T 30b4d9cc324f47d28c5d84bf03e91390 P 3d02a3b7b1384997b48d3255f708864c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:40:58.453578 15739 ts_tablet_manager.cc:1905] T 30b4d9cc324f47d28c5d84bf03e91390 P edff6837b14e4df1afe5b23c274ab44e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:40:58.457531 15709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:58.438301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49236 (local address 127.24.76.65:42507)
0808 11:40:58.438485 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:58.438491 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:58.438599 (+ 108us) server_negotiation.cc:407] Connection header received
0808 11:40:58.438773 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:58.438779 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:58.438882 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:58.438986 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:58.440825 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:58.442086 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:58.444091 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:58.444790 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:58.444918 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:58.449159 (+ 4241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:58.449171 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:58.449187 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:58.449231 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:58.452917 (+ 3686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:58.453997 (+ 1080us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:58.454007 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:58.454010 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:58.454167 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:58.454739 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:58.454745 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:58.454748 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:58.455316 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:58.455638 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:58.456906 (+ 1268us) server_negotiation.cc:299] Negotiation successful
0808 11:40:58.457154 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 11:40:58.457858 15740 ts_tablet_manager.cc:1918] T 30b4d9cc324f47d28c5d84bf03e91390 P d77df0c06c4b49288f5d8feddb6e9691: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:40:58.458065 15740 log.cc:1196] T 30b4d9cc324f47d28c5d84bf03e91390 P d77df0c06c4b49288f5d8feddb6e9691: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/wal/wals/30b4d9cc324f47d28c5d84bf03e91390
I20260808 11:40:58.458495 15295 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:40:58.461472 15739 ts_tablet_manager.cc:1918] T 30b4d9cc324f47d28c5d84bf03e91390 P edff6837b14e4df1afe5b23c274ab44e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:40:58.461647 15739 log.cc:1196] T 30b4d9cc324f47d28c5d84bf03e91390 P edff6837b14e4df1afe5b23c274ab44e: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/wal/wals/30b4d9cc324f47d28c5d84bf03e91390
I20260808 11:40:58.462277 15739 ts_tablet_manager.cc:1939] T 30b4d9cc324f47d28c5d84bf03e91390 P edff6837b14e4df1afe5b23c274ab44e: Deleting consensus metadata
I20260808 11:40:58.464267 14990 catalog_manager.cc:4841] TS edff6837b14e4df1afe5b23c274ab44e (127.24.76.66:43777): tablet 30b4d9cc324f47d28c5d84bf03e91390 (table test-table [id=2c62ff700f0d4ef0a5885f4715449fec]) successfully deleted
I20260808 11:40:58.466076 15741 ts_tablet_manager.cc:1918] T 30b4d9cc324f47d28c5d84bf03e91390 P 3d02a3b7b1384997b48d3255f708864c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:40:58.466504 15741 log.cc:1196] T 30b4d9cc324f47d28c5d84bf03e91390 P 3d02a3b7b1384997b48d3255f708864c: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/wal/wals/30b4d9cc324f47d28c5d84bf03e91390
I20260808 11:40:58.467489 15741 ts_tablet_manager.cc:1939] T 30b4d9cc324f47d28c5d84bf03e91390 P 3d02a3b7b1384997b48d3255f708864c: Deleting consensus metadata
I20260808 11:40:58.469287 14992 catalog_manager.cc:4841] TS 3d02a3b7b1384997b48d3255f708864c (127.24.76.67:35923): tablet 30b4d9cc324f47d28c5d84bf03e91390 (table test-table [id=2c62ff700f0d4ef0a5885f4715449fec]) successfully deleted
I20260808 11:40:58.551939 15740 ts_tablet_manager.cc:1939] T 30b4d9cc324f47d28c5d84bf03e91390 P d77df0c06c4b49288f5d8feddb6e9691: Deleting consensus metadata
I20260808 11:40:58.554234 14996 catalog_manager.cc:4841] TS d77df0c06c4b49288f5d8feddb6e9691 (127.24.76.65:42507): tablet 30b4d9cc324f47d28c5d84bf03e91390 (table test-table [id=2c62ff700f0d4ef0a5885f4715449fec]) successfully deleted
I20260808 11:40:58.554349 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 15073
I20260808 11:40:58.585620 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 15308
I20260808 11:40:58.600886 15706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:58.586569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60790 (local address 127.24.76.66:43777)
0808 11:40:58.586726 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:58.586731 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:58.586749 (+ 18us) server_negotiation.cc:407] Connection header received
0808 11:40:58.586891 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:58.586895 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:58.586957 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:58.587039 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:58.588736 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:58.589471 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:58.591355 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:58.591827 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:58.591918 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:58.595184 (+ 3266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:58.595199 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:58.595214 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:58.595258 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:58.598001 (+ 2743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:58.598768 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:58.598778 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:58.598798 (+ 20us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:58.598956 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:58.599359 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:58.599362 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:58.599364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:58.599703 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:58.600000 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:58.600319 (+ 319us) server_negotiation.cc:299] Negotiation successful
0808 11:40:58.600542 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 11:40:58.601922 15495 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:40:58.690023 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 15308
I20260808 11:40:58.714121 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 15506
I20260808 11:40:58.735054 15710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:58.715054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60780 (local address 127.24.76.67:35923)
0808 11:40:58.715277 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 11:40:58.715287 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:40:58.715335 (+ 48us) server_negotiation.cc:407] Connection header received
0808 11:40:58.715416 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:40:58.715422 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:40:58.715513 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:40:58.715625 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:40:58.716934 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:58.718183 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:58.719519 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:40:58.720203 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:40:58.720393 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:40:58.727962 (+ 7569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:40:58.727987 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:40:58.728009 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:40:58.728059 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:40:58.732037 (+ 3978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:58.732721 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:58.732732 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:58.732735 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:58.732931 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:40:58.733267 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:40:58.733272 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:40:58.733275 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:40:58.733825 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:40:58.734213 (+ 388us) server_negotiation.cc:1035] Waiting for connection context
0808 11:40:58.734391 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 11:40:58.734621 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 11:40:58.735905 15675 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:40:58.861384 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 15506
I20260808 11:40:58.887020 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 14964
I20260808 11:40:58.888113 15063 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:40:59.000186 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 14964
I20260808 11:40:59.027540 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:36973
--webserver_interface=127.24.76.126
--webserver_port=33745
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:34219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:36973
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:40:59.358546 15749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:40:59.358809 15749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:40:59.358937 15749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:40:59.365939 15749 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:40:59.366034 15749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:40:59.366106 15749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:40:59.366173 15749 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:40:59.366231 15749 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:40:59.375547 15749 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:34219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:36973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:36973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=33745
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:40:59.376824 15749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:40:59.378132 15749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:40:59.389127 15755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:59.389247 15758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:59.389211 15756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:40:59.390208 15749 server_base.cc:1034] running on GCE node
I20260808 11:40:59.390976 15749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:40:59.392251 15749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:40:59.393462 15749 hybrid_clock.cc:648] HybridClock initialized: now 1786189259393389 us; error 63 us; skew 500 ppm
I20260808 11:40:59.393782 15749 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:40:59 dist-test-slave-tc2s krb5kdc[14111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189259, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:40:59.396875 15749 init.cc:377] Logged in from keytab as kudu/127.24.76.126@KRBTEST.COM (short username kudu)
I20260808 11:40:59.398277 15749 webserver.cc:460] Webserver started at http://127.24.76.126:33745/ using document root <none> and password file <none>
I20260808 11:40:59.398891 15749 fs_manager.cc:359] Metadata directory not provided
I20260808 11:40:59.398991 15749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:40:59.404373 15749 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 11:40:59.407912 15767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:59.409476 15749 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 11:40:59.409627 15749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "458b6770d36f44afb76ef710569715d2"
format_stamp: "Formatted at 2026-08-08 11:40:53 on dist-test-slave-tc2s"
server_key: "3dcafd6cb11bcf112adf724879de3f73"
server_key_iv: "d413c28bccec335c3d166ef34022840d"
server_key_version: "encryptionkey@0"
I20260808 11:40:59.410858 15749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:40:59.456317 15749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:40:59.460707 15749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:40:59.460983 15749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:40:59.478319 15749 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:36973
I20260808 11:40:59.478338 15848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:36973 every 8 connection(s)
I20260808 11:40:59.479735 15749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:40:59.482331 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 15749
I20260808 11:40:59.483350 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.65:42507
--local_ip_for_outbound_sockets=127.24.76.65
--tserver_master_addrs=127.24.76.126:36973
--webserver_port=44759
--webserver_interface=127.24.76.65
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:34219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:40:59.487922 15849 sys_catalog.cc:263] Verifying existing consensus state
I20260808 11:40:59.492086 15849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2: Bootstrap starting.
I20260808 11:40:59.512181 15849 log.cc:824] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2: Log is configured to *not* fsync() on all Append() calls
I20260808 11:40:59.527881 15849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 11:40:59.528376 15849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2: Bootstrap complete.
I20260808 11:40:59.537534 15849 raft_consensus.cc:348] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "458b6770d36f44afb76ef710569715d2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 36973 } }
I20260808 11:40:59.538447 15849 raft_consensus.cc:729] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 458b6770d36f44afb76ef710569715d2, State: Initialized, Role: FOLLOWER
I20260808 11:40:59.539021 15849 consensus_queue.cc:260] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "458b6770d36f44afb76ef710569715d2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 36973 } }
I20260808 11:40:59.539243 15849 raft_consensus.cc:388] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:40:59.539350 15849 raft_consensus.cc:482] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:40:59.539508 15849 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [term 2 FOLLOWER]: Advancing to term 3
I20260808 11:40:59.543363 15849 raft_consensus.cc:504] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "458b6770d36f44afb76ef710569715d2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 36973 } }
I20260808 11:40:59.543829 15849 leader_election.cc:302] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 458b6770d36f44afb76ef710569715d2; no voters:
I20260808 11:40:59.544428 15849 leader_election.cc:288] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 11:40:59.544592 15858 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 11:40:59.546161 15858 raft_consensus.cc:686] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [term 3 LEADER]: Becoming Leader. State: Replica: 458b6770d36f44afb76ef710569715d2, State: Running, Role: LEADER
I20260808 11:40:59.546514 15849 sys_catalog.cc:564] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:40:59.546759 15858 consensus_queue.cc:237] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "458b6770d36f44afb76ef710569715d2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 36973 } }
I20260808 11:40:59.551473 15861 sys_catalog.cc:455] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 458b6770d36f44afb76ef710569715d2. Latest consensus state: current_term: 3 leader_uuid: "458b6770d36f44afb76ef710569715d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "458b6770d36f44afb76ef710569715d2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 36973 } } }
I20260808 11:40:59.551867 15861 sys_catalog.cc:458] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [sys.catalog]: This master's current role is: LEADER
I20260808 11:40:59.552378 15859 sys_catalog.cc:455] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "458b6770d36f44afb76ef710569715d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "458b6770d36f44afb76ef710569715d2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 36973 } } }
I20260808 11:40:59.552654 15859 sys_catalog.cc:458] T 00000000000000000000000000000000 P 458b6770d36f44afb76ef710569715d2 [sys.catalog]: This master's current role is: LEADER
I20260808 11:40:59.554781 15872 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:40:59.568010 15872 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:40:59.569518 15872 catalog_manager.cc:1259] Loaded cluster ID: 56e1e86b3cc14593a8d52c78152b399d
I20260808 11:40:59.569679 15872 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 11:40:59.572230 15872 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f905662e9ad google::LogMessage::Fail() at ??:0
@ 0x7f905663188a google::LogMessage::SendToLog() at ??:0
@ 0x7f905662e4de google::LogMessage::Flush() at ??:0
@ 0x7f905662fa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f906c061512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f906c05e976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f906c0e6610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f906c0e6439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f905927a345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f905927d91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f905927d799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f9059258eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f905c5326db start_thread at ??:0
@ 0x7f90554c861f clone at ??:0
W20260808 11:40:59.872594 15853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:40:59.872859 15853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:40:59.872987 15853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:40:59.880139 15853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:40:59.880237 15853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:40:59.880406 15853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.65
I20260808 11:40:59.889734 15853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:34219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.65:42507
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.76.65
--webserver_port=44759
--tserver_master_addrs=127.24.76.126:36973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.65
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:40:59.890964 15853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:40:59.892277 15853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:40:59.904731 15917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:59.904811 15918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:40:59.904779 15920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:40:59.905561 15853 server_base.cc:1034] running on GCE node
I20260808 11:40:59.906133 15853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:40:59.906965 15853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:40:59.908154 15853 hybrid_clock.cc:648] HybridClock initialized: now 1786189259908112 us; error 31 us; skew 500 ppm
I20260808 11:40:59.908452 15853 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:40:59 dist-test-slave-tc2s krb5kdc[14111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189259, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:40:59.911473 15853 init.cc:377] Logged in from keytab as kudu/127.24.76.65@KRBTEST.COM (short username kudu)
I20260808 11:40:59.912777 15853 webserver.cc:460] Webserver started at http://127.24.76.65:44759/ using document root <none> and password file <none>
I20260808 11:40:59.913422 15853 fs_manager.cc:359] Metadata directory not provided
I20260808 11:40:59.913548 15853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:40:59.918524 15853 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:40:59.923036 15927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:59.924198 15853 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 11:40:59.924347 15853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/wal
uuid: "d77df0c06c4b49288f5d8feddb6e9691"
format_stamp: "Formatted at 2026-08-08 11:40:54 on dist-test-slave-tc2s"
server_key: "1dd4a2a594b34009141005e2ea168904"
server_key_iv: "de79f555936eb8ac8c0d68a0a070a8eb"
server_key_version: "encryptionkey@0"
I20260808 11:40:59.925297 15853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:40:59.937515 15853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:40:59.941054 15853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:40:59.941345 15853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:40:59.942284 15853 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:40:59.943538 15853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:40:59.943635 15853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:59.943737 15853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:40:59.943807 15853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:40:59.974478 15853 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.65:42507
I20260808 11:40:59.974550 16090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.65:42507 every 8 connection(s)
I20260808 11:40:59.975836 15853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
I20260808 11:40:59.980464 16093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:59.978659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.65:40629)
0808 11:40:59.979387 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:59.979604 (+ 217us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":285,"threads_started":1}
W20260808 11:40:59.981081 16091 heartbeater.cc:644] Failed to heartbeat to 127.24.76.126:36973 (0 consecutive failures): Network error: Failed to ping master at 127.24.76.126:36973: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
I20260808 11:40:59.981796 16093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:59.981493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.65:43637)
0808 11:40:59.981626 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:59.981692 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 11:40:59.982609 16093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:40:59.982299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.65:38563)
0808 11:40:59.982430 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 11:40:59.982503 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 11:40:59.982836 16091 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 11:40:59.983565 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 15853
I20260808 11:40:59.984159 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.66:43777
--local_ip_for_outbound_sockets=127.24.76.66
--tserver_master_addrs=127.24.76.126:36973
--webserver_port=34609
--webserver_interface=127.24.76.66
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:34219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:41:00.308966 16096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:41:00.309238 16096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:41:00.309366 16096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:41:00.316494 16096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:41:00.316594 16096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:41:00.316741 16096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.66
I20260808 11:41:00.326192 16096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:34219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.66:43777
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.76.66
--webserver_port=34609
--tserver_master_addrs=127.24.76.126:36973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.66
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:41:00.327399 16096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:41:00.328687 16096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:41:00.340756 16104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:41:00.340801 16105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:41:00.341152 16107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:41:00.342057 16096 server_base.cc:1034] running on GCE node
I20260808 11:41:00.342613 16096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:41:00.343443 16096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:41:00.344645 16096 hybrid_clock.cc:648] HybridClock initialized: now 1786189260344599 us; error 35 us; skew 500 ppm
I20260808 11:41:00.344933 16096 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:41:00 dist-test-slave-tc2s krb5kdc[14111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189260, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:41:00.347852 16096 init.cc:377] Logged in from keytab as kudu/127.24.76.66@KRBTEST.COM (short username kudu)
I20260808 11:41:00.349185 16096 webserver.cc:460] Webserver started at http://127.24.76.66:34609/ using document root <none> and password file <none>
I20260808 11:41:00.349813 16096 fs_manager.cc:359] Metadata directory not provided
I20260808 11:41:00.349964 16096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:41:00.354919 16096 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 11:41:00.357435 16114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:41:00.358696 16096 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 11:41:00.358842 16096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/wal
uuid: "edff6837b14e4df1afe5b23c274ab44e"
format_stamp: "Formatted at 2026-08-08 11:40:54 on dist-test-slave-tc2s"
server_key: "0c56724677c9398dc750b305ffa0b3c7"
server_key_iv: "722b13ef4d67a0ef14f375f71f1c74b8"
server_key_version: "encryptionkey@0"
I20260808 11:41:00.359779 16096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:41:00.372970 16096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:41:00.376523 16096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:41:00.376763 16096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:41:00.377693 16096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:41:00.379061 16096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:41:00.379163 16096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:41:00.379264 16096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:41:00.379369 16096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:41:00.409206 16096 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.66:43777
I20260808 11:41:00.409288 16288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.66:43777 every 8 connection(s)
I20260808 11:41:00.410593 16096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
I20260808 11:41:00.415118 16291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:00.413383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.66:47779)
0808 11:41:00.414115 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:00.414328 (+ 213us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":260,"threads_started":1}
W20260808 11:41:00.415804 16289 heartbeater.cc:644] Failed to heartbeat to 127.24.76.126:36973 (0 consecutive failures): Network error: Failed to ping master at 127.24.76.126:36973: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
I20260808 11:41:00.416525 16291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:00.416236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.66:45577)
0808 11:41:00.416358 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:00.416423 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 11:41:00.417224 16291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:00.416946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.66:42711)
0808 11:41:00.417047 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:00.417116 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 11:41:00.417454 16289 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 11:41:00.419761 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 16096
I20260808 11:41:00.420367 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.67:35923
--local_ip_for_outbound_sockets=127.24.76.67
--tserver_master_addrs=127.24.76.126:36973
--webserver_port=34563
--webserver_interface=127.24.76.67
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:34219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:41:00.744271 16292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:41:00.744511 16292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:41:00.744642 16292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:41:00.751629 16292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:41:00.751734 16292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:41:00.751883 16292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.67
I20260808 11:41:00.761139 16292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:34219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.67:35923
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.76.67
--webserver_port=34563
--tserver_master_addrs=127.24.76.126:36973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.67
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:41:00.762315 16292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:41:00.763540 16292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:41:00.775689 16299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:41:00.775646 16302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:41:00.775727 16300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:41:00.776631 16292 server_base.cc:1034] running on GCE node
I20260808 11:41:00.777210 16292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:41:00.778131 16292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:41:00.779325 16292 hybrid_clock.cc:648] HybridClock initialized: now 1786189260779279 us; error 40 us; skew 500 ppm
I20260808 11:41:00.779615 16292 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:41:00 dist-test-slave-tc2s krb5kdc[14111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189260, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:41:00.782573 16292 init.cc:377] Logged in from keytab as kudu/127.24.76.67@KRBTEST.COM (short username kudu)
I20260808 11:41:00.783860 16292 webserver.cc:460] Webserver started at http://127.24.76.67:34563/ using document root <none> and password file <none>
I20260808 11:41:00.784478 16292 fs_manager.cc:359] Metadata directory not provided
I20260808 11:41:00.784608 16292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:41:00.790591 16292 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260808 11:41:00.793134 16311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:41:00.794459 16292 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 11:41:00.794617 16292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/wal
uuid: "3d02a3b7b1384997b48d3255f708864c"
format_stamp: "Formatted at 2026-08-08 11:40:55 on dist-test-slave-tc2s"
server_key: "9e705ee2ee6732036a0f3465c60d9873"
server_key_iv: "611ad4bceec41c52e7f0cdd1e38a1bc0"
server_key_version: "encryptionkey@0"
I20260808 11:41:00.795552 16292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:41:00.815861 16292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:41:00.819679 16292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:41:00.819924 16292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:41:00.820879 16292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:41:00.822186 16292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:41:00.822295 16292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:41:00.822405 16292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:41:00.822476 16292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:41:00.851722 16292 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.67:35923
I20260808 11:41:00.851776 16484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.67:35923 every 8 connection(s)
I20260808 11:41:00.853132 16292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
I20260808 11:41:00.856143 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 16292
I20260808 11:41:00.858155 16488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:00.855958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.67:51413)
0808 11:41:00.856786 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:00.857078 (+ 292us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":294,"threads_started":1}
W20260808 11:41:00.858825 16485 heartbeater.cc:644] Failed to heartbeat to 127.24.76.126:36973 (0 consecutive failures): Network error: Failed to ping master at 127.24.76.126:36973: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
I20260808 11:41:00.859576 16488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:00.859270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.67:42539)
0808 11:41:00.859402 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:00.859467 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 11:41:00.860351 16488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:00.860031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.67:50691)
0808 11:41:00.860165 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:00.860235 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 11:41:00.860553 16485 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 11:41:00.984090 16490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:00.983292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.65:33057)
0808 11:41:00.983736 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:00.983846 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":210,"threads_started":1}
I20260808 11:41:01.418723 16491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:01.417956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.66:42395)
0808 11:41:01.418388 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:01.418507 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":197,"threads_started":1}
I20260808 11:41:01.861922 16492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:01.861067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.67:40699)
0808 11:41:01.861511 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:01.861606 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":224,"threads_started":1}
I20260808 11:41:01.985587 16494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:01.984874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.65:53853)
0808 11:41:01.985344 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:01.985426 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":236,"threads_started":1}
I20260808 11:41:02.420189 16495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:02.419502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.66:37979)
0808 11:41:02.419943 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:02.420028 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":225,"threads_started":1}
I20260808 11:41:02.863462 16497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:02.862731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.67:36833)
0808 11:41:02.863200 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:02.863285 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":236,"threads_started":1}
I20260808 11:41:02.986994 16498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:02.986319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.65:54845)
0808 11:41:02.986756 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:02.986833 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":237,"threads_started":1}
I20260808 11:41:03.421607 16499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:03.420954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.66:51719)
0808 11:41:03.421369 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:03.421447 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":202,"threads_started":1}
I20260808 11:41:03.864880 16500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:03.864205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.67:49511)
0808 11:41:03.864629 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:03.864707 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":220,"threads_started":1}
I20260808 11:41:03.988500 16501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:03.987811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.65:57261)
0808 11:41:03.988227 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:03.988306 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":208,"threads_started":1}
I20260808 11:41:04.423035 16502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:04.422337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.66:54443)
0808 11:41:04.422765 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:04.422845 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":219,"threads_started":1}
I20260808 11:41:04.866345 16503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:04.865643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.67:36123)
0808 11:41:04.866085 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:04.866163 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":203,"threads_started":1}
I20260808 11:41:04.990077 16504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:04.989321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.65:59613)
0808 11:41:04.989782 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:04.989861 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":239,"threads_started":1}
I20260808 11:41:05.424504 16506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:05.423809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.66:43315)
0808 11:41:05.424264 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:05.424345 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":231,"threads_started":1}
I20260808 11:41:05.867933 16508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:05.867181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.67:53115)
0808 11:41:05.867686 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:05.867762 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":268,"threads_started":1}
I20260808 11:41:05.991528 16509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:05.990856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.65:37295)
0808 11:41:05.991286 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:05.991372 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":217,"threads_started":1}
I20260808 11:41:06.426005 16513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:06.425267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.66:51465)
0808 11:41:06.425712 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:06.425804 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":217,"threads_started":1}
I20260808 11:41:06.869392 16514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:06.868710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.67:43543)
0808 11:41:06.869148 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:06.869236 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":226,"threads_started":1}
I20260808 11:41:06.993120 16515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:06.992357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.65:53653)
0808 11:41:06.992814 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:06.992895 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":199,"threads_started":1}
I20260808 11:41:07.427637 16520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:07.426850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.66:43603)
0808 11:41:07.427342 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:07.427429 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":200,"threads_started":1}
I20260808 11:41:07.870990 16521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:07.870203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.67:32913)
0808 11:41:07.870685 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:07.870766 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":211,"threads_started":1}
I20260808 11:41:07.994714 16522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:07.994002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.65:40957)
0808 11:41:07.994467 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:07.994550 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":242,"threads_started":1}
I20260808 11:41:08.429325 16523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:08.428516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.66:33875)
0808 11:41:08.429083 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:08.429164 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":330,"threads_started":1}
I20260808 11:41:08.872454 16524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:08.871779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.67:49773)
0808 11:41:08.872220 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:08.872298 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":233,"threads_started":1}
I20260808 11:41:08.996269 16525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:08.995526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.65:33657)
0808 11:41:08.996018 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:08.996096 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":236,"threads_started":1}
I20260808 11:41:09.430878 16526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:09.430154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.66:56683)
0808 11:41:09.430621 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:09.430704 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":213,"threads_started":1}
I20260808 11:41:09.873960 16527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:09.873208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.67:39985)
0808 11:41:09.873669 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:09.873750 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":217,"threads_started":1}
I20260808 11:41:09.997915 16529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:09.997122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.65:56987)
0808 11:41:09.997578 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:09.997684 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":246,"threads_started":1}
I20260808 11:41:10.432402 16530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:10.431736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.66:57221)
0808 11:41:10.432164 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:10.432244 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":229,"threads_started":1}
I20260808 11:41:10.875443 16531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:10.874758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.67:47817)
0808 11:41:10.875198 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:10.875277 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":252,"threads_started":1}
I20260808 11:41:10.999475 16533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:10.998734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.65:60123)
0808 11:41:10.999190 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:10.999283 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":243,"threads_started":1}
I20260808 11:41:11.433925 16534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:11.433187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.66:43015)
0808 11:41:11.433637 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:11.433730 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":236,"threads_started":1}
I20260808 11:41:11.876999 16537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:11.876265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.67:39865)
0808 11:41:11.876709 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:11.876802 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":236,"threads_started":1}
I20260808 11:41:12.000969 16538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:12.000323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.65:50625)
0808 11:41:12.000740 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:12.000820 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":210,"threads_started":1}
I20260808 11:41:12.435415 16539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:12.434718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.66:38223)
0808 11:41:12.435169 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:12.435250 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":218,"threads_started":1}
I20260808 11:41:12.878566 16541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:12.877795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.67:60251)
0808 11:41:12.878266 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:12.878378 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":262,"threads_started":1}
I20260808 11:41:13.002490 16543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:13.001738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.65:53199)
0808 11:41:13.002215 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:13.002304 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":246,"threads_started":1}
I20260808 11:41:13.436939 16544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:13.436226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.66:54485)
0808 11:41:13.436642 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:13.436733 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":189,"threads_started":1}
I20260808 11:41:13.880084 16545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:13.879337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.67:36153)
0808 11:41:13.879806 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:13.879895 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":230,"threads_started":1}
I20260808 11:41:14.003995 16546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:14.003290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.65:53677)
0808 11:41:14.003724 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:14.003803 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":226,"threads_started":1}
I20260808 11:41:14.438390 16547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:14.437693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.66:50727)
0808 11:41:14.438140 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:14.438218 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":246,"threads_started":1}
I20260808 11:41:14.881601 16548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:14.880862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.67:46247)
0808 11:41:14.881329 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:14.881408 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":255,"threads_started":1}
I20260808 11:41:15.005823 16550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:15.005066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.65:41649)
0808 11:41:15.005552 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:15.005639 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":272,"threads_started":1}
I20260808 11:41:15.440001 16552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:15.439335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.66:59367)
0808 11:41:15.439721 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:15.439816 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":194,"threads_started":1}
I20260808 11:41:15.859386 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 15853
I20260808 11:41:15.876206 16553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:15.860429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38908 (local address 127.24.76.65:42507)
0808 11:41:15.861032 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0808 11:41:15.861048 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 11:41:15.861087 (+ 39us) server_negotiation.cc:407] Connection header received
0808 11:41:15.862760 (+ 1673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:41:15.862770 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:41:15.863235 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:41:15.863595 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:41:15.864920 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:15.866223 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:15.867439 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:15.868094 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:15.868225 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:15.870766 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:41:15.870774 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:41:15.870788 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:41:15.870827 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:41:15.873536 (+ 2709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:15.874173 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:15.874179 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:15.874186 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:15.874347 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:15.874632 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:15.874635 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:15.874637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:15.875097 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:41:15.875332 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0808 11:41:15.875633 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 11:41:15.875932 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":270,"threads_started":1}
I20260808 11:41:15.877504 16083 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:41:15.883384 16556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:15.882694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:36973 (local address 127.24.76.67:48579)
0808 11:41:15.883116 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:15.883218 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:36973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":221,"threads_started":1}
I20260808 11:41:15.959371 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 15853
I20260808 11:41:15.981194 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 16096
I20260808 11:41:15.997725 16560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:15.982223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57594 (local address 127.24.76.66:43777)
0808 11:41:15.982760 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0808 11:41:15.982774 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 11:41:15.982813 (+ 39us) server_negotiation.cc:407] Connection header received
0808 11:41:15.984424 (+ 1611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:41:15.984434 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:41:15.984872 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:41:15.985151 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:41:15.986566 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:15.987771 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:15.988919 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:15.989578 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:15.989704 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:15.992018 (+ 2314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:41:15.992025 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:41:15.992039 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:41:15.992078 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:41:15.994991 (+ 2913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:15.995605 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:15.995611 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:15.995617 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:15.995775 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:15.996055 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:15.996058 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:15.996060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:15.996548 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:41:15.996778 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0808 11:41:15.997068 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 11:41:15.997438 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":266,"threads_started":1}
I20260808 11:41:15.999046 16284 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:41:16.080550 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 16096
I20260808 11:41:16.101759 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 16292
I20260808 11:41:16.118854 16563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:16.102715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55550 (local address 127.24.76.67:35923)
0808 11:41:16.103313 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0808 11:41:16.103327 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 11:41:16.103371 (+ 44us) server_negotiation.cc:407] Connection header received
0808 11:41:16.105031 (+ 1660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:41:16.105041 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:41:16.105482 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:41:16.105915 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:41:16.107516 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:16.108713 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:16.109845 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:16.110632 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:16.110769 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:16.113182 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:41:16.113190 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:41:16.113204 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:41:16.113240 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:41:16.116043 (+ 2803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:16.116656 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:16.116662 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:16.116668 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:16.116826 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:16.117147 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:16.117150 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:16.117152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:16.117642 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:41:16.117884 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0808 11:41:16.118243 (+ 359us) server_negotiation.cc:299] Negotiation successful
0808 11:41:16.118565 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":288,"threads_started":1}
I20260808 11:41:16.120177 16476 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:41:16.202468 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 16292
2026-08-08T11:41:16Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24355 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:41:16 dist-test-slave-tc2s krb5kdc[16567](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:41:16 dist-test-slave-tc2s krb5kdc[16567](info): set up 2 sockets
Aug 08 11:41:16 dist-test-slave-tc2s krb5kdc[16567](info): commencing operation
krb5kdc: starting...
W20260808 11:41:17.419054 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:41:17 dist-test-slave-tc2s krb5kdc[16567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:41:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:41:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
I20260808 11:41:17.687690 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:33899
--webserver_interface=127.24.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:33281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:33899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:41:18.015514 16590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:41:18.015780 16590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:41:18.015897 16590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:41:18.022848 16590 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 11:41:18.022939 16590 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 11:41:18.023000 16590 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 11:41:18.023073 16590 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:41:18.023142 16590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:41:18.023236 16590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:41:18.023288 16590 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:41:18.023365 16590 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:41:18.032652 16590 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:33281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:33899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:33899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:41:18.033802 16590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:41:18.035111 16590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:41:18.047380 16599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:41:18.047351 16600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:41:18.047492 16603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:41:18.048792 16590 server_base.cc:1034] running on GCE node
I20260808 11:41:18.049753 16590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:41:18.051616 16590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:41:18.052836 16590 hybrid_clock.cc:648] HybridClock initialized: now 1786189278052765 us; error 48 us; skew 500 ppm
I20260808 11:41:18.053153 16590 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:41:18 dist-test-slave-tc2s krb5kdc[16567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189278, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:41:18.056102 16590 init.cc:377] Logged in from keytab as kudu/127.24.76.126@KRBTEST.COM (short username kudu)
I20260808 11:41:18.057564 16590 webserver.cc:460] Webserver started at http://127.24.76.126:36373/ using document root <none> and password file <none>
I20260808 11:41:18.058259 16590 fs_manager.cc:359] Metadata directory not provided
I20260808 11:41:18.058362 16590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:41:18.058661 16590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:41:18.061503 16590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "861318038ac14ba6acca16718dbd7fb6"
format_stamp: "Formatted at 2026-08-08 11:41:18 on dist-test-slave-tc2s"
server_key: "f5a9b17562f9d7e52d51fed9058ecc97"
server_key_iv: "d5caf735e621b5375e1ba3cf8385355d"
server_key_version: "encryptionkey@0"
I20260808 11:41:18.062265 16590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "861318038ac14ba6acca16718dbd7fb6"
format_stamp: "Formatted at 2026-08-08 11:41:18 on dist-test-slave-tc2s"
server_key: "f5a9b17562f9d7e52d51fed9058ecc97"
server_key_iv: "d5caf735e621b5375e1ba3cf8385355d"
server_key_version: "encryptionkey@0"
I20260808 11:41:18.067097 16590 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 11:41:18.070353 16616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:41:18.071525 16590 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 11:41:18.071669 16590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "861318038ac14ba6acca16718dbd7fb6"
format_stamp: "Formatted at 2026-08-08 11:41:18 on dist-test-slave-tc2s"
server_key: "f5a9b17562f9d7e52d51fed9058ecc97"
server_key_iv: "d5caf735e621b5375e1ba3cf8385355d"
server_key_version: "encryptionkey@0"
I20260808 11:41:18.071856 16590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:41:18.088016 16590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:41:18.091527 16590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:41:18.091751 16590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:41:18.108618 16590 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:33899
I20260808 11:41:18.108639 16681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:33899 every 8 connection(s)
I20260808 11:41:18.109829 16590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:41:18.112187 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 16590
I20260808 11:41:18.112426 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:41:18.113233 24881 external_mini_cluster.cc:1428] Setting key df839b5f48d3fdcf077bd4f32fa4e6bd
I20260808 11:41:18.113921 16685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:41:18.127416 16685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6: Bootstrap starting.
Aug 08 11:41:18 dist-test-slave-tc2s krb5kdc[16567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:41:18.130832 16685 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6: Neither blocks nor log segments found. Creating new log.
I20260808 11:41:18.131930 16685 log.cc:824] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6: Log is configured to *not* fsync() on all Append() calls
I20260808 11:41:18.134964 16685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6: No bootstrap required, opened a new log
I20260808 11:41:18.136801 16689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:18.115257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43034 (local address 127.24.76.126:33899)
0808 11:41:18.115975 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0808 11:41:18.115988 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 11:41:18.116027 (+ 39us) server_negotiation.cc:407] Connection header received
0808 11:41:18.118314 (+ 2287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:41:18.118335 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:41:18.118696 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:41:18.119009 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:41:18.120833 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:18.122147 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:18.124026 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:18.124688 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:18.124812 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:18.129667 (+ 4855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:41:18.129676 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:41:18.129689 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:41:18.129722 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:41:18.132900 (+ 3178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:18.133680 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:18.133686 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:18.133692 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:18.133852 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:18.134277 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:18.134280 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:18.134282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:18.134773 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:41:18.134997 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0808 11:41:18.135478 (+ 481us) server_negotiation.cc:299] Negotiation successful
0808 11:41:18.135798 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":339,"threads_started":1}
I20260808 11:41:18.142418 16685 raft_consensus.cc:348] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861318038ac14ba6acca16718dbd7fb6" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 33899 } }
I20260808 11:41:18.142748 16685 raft_consensus.cc:374] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:41:18.142840 16685 raft_consensus.cc:729] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 861318038ac14ba6acca16718dbd7fb6, State: Initialized, Role: FOLLOWER
I20260808 11:41:18.143326 16685 consensus_queue.cc:260] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861318038ac14ba6acca16718dbd7fb6" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 33899 } }
I20260808 11:41:18.143533 16685 raft_consensus.cc:388] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:41:18.143657 16685 raft_consensus.cc:482] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:41:18.143803 16685 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:41:18.145794 16685 raft_consensus.cc:504] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861318038ac14ba6acca16718dbd7fb6" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 33899 } }
I20260808 11:41:18.146250 16685 leader_election.cc:302] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 861318038ac14ba6acca16718dbd7fb6; no voters:
I20260808 11:41:18.146801 16685 leader_election.cc:288] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:41:18.146891 16691 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:41:18.147540 16691 raft_consensus.cc:686] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [term 1 LEADER]: Becoming Leader. State: Replica: 861318038ac14ba6acca16718dbd7fb6, State: Running, Role: LEADER
I20260808 11:41:18.148099 16691 consensus_queue.cc:237] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861318038ac14ba6acca16718dbd7fb6" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 33899 } }
I20260808 11:41:18.148876 16685 sys_catalog.cc:564] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:41:18.151558 16692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "861318038ac14ba6acca16718dbd7fb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861318038ac14ba6acca16718dbd7fb6" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 33899 } } }
I20260808 11:41:18.151731 16694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 861318038ac14ba6acca16718dbd7fb6. Latest consensus state: current_term: 1 leader_uuid: "861318038ac14ba6acca16718dbd7fb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861318038ac14ba6acca16718dbd7fb6" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 33899 } } }
I20260808 11:41:18.151929 16692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [sys.catalog]: This master's current role is: LEADER
I20260808 11:41:18.151988 16694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [sys.catalog]: This master's current role is: LEADER
I20260808 11:41:18.153822 16705 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:41:18.158192 16705 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:41:18.165607 16705 catalog_manager.cc:1347] Generated new cluster ID: b3ccdb898f894d9faccd4d38e799eb5d
I20260808 11:41:18.165704 16705 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:41:18.181311 16705 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:41:18.182605 16705 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:41:18.191440 16705 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6: Generated new TSK 0
I20260808 11:41:18.192209 16705 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab.
Entry for principal kudu/127.24.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab.
I20260808 11:41:18.262457 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.65:0
--local_ip_for_outbound_sockets=127.24.76.65
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:33899
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:33281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:41:18.590665 16726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:41:18.590940 16726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:41:18.591090 16726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:41:18.597988 16726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:41:18.598099 16726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:41:18.598249 16726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.65
I20260808 11:41:18.607561 16726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:33281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:33899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.65
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:41:18.608700 16726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:41:18.609923 16726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:41:18.622382 16737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:41:18.622335 16742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:41:18.622340 16738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:41:18.623486 16726 server_base.cc:1034] running on GCE node
I20260808 11:41:18.624063 16726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:41:18.624963 16726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:41:18.626150 16726 hybrid_clock.cc:648] HybridClock initialized: now 1786189278626103 us; error 38 us; skew 500 ppm
I20260808 11:41:18.626428 16726 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:41:18 dist-test-slave-tc2s krb5kdc[16567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189278, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:41:18.632254 16726 init.cc:377] Logged in from keytab as kudu/127.24.76.65@KRBTEST.COM (short username kudu)
I20260808 11:41:18.633651 16726 webserver.cc:460] Webserver started at http://127.24.76.65:35835/ using document root <none> and password file <none>
I20260808 11:41:18.634338 16726 fs_manager.cc:359] Metadata directory not provided
I20260808 11:41:18.634467 16726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:41:18.634757 16726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:41:18.637617 16726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/data/instance:
uuid: "f07b914ccc874645a5998194df57f1cc"
format_stamp: "Formatted at 2026-08-08 11:41:18 on dist-test-slave-tc2s"
server_key: "299ecc852022c57d5f1043b1eed43ca4"
server_key_iv: "020cb5d709c4d261c66596cf43dc0b9f"
server_key_version: "encryptionkey@0"
I20260808 11:41:18.638391 16726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance:
uuid: "f07b914ccc874645a5998194df57f1cc"
format_stamp: "Formatted at 2026-08-08 11:41:18 on dist-test-slave-tc2s"
server_key: "299ecc852022c57d5f1043b1eed43ca4"
server_key_iv: "020cb5d709c4d261c66596cf43dc0b9f"
server_key_version: "encryptionkey@0"
I20260808 11:41:18.643271 16726 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 11:41:18.646467 16753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:41:18.647683 16726 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:41:18.647831 16726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/wal
uuid: "f07b914ccc874645a5998194df57f1cc"
format_stamp: "Formatted at 2026-08-08 11:41:18 on dist-test-slave-tc2s"
server_key: "299ecc852022c57d5f1043b1eed43ca4"
server_key_iv: "020cb5d709c4d261c66596cf43dc0b9f"
server_key_version: "encryptionkey@0"
I20260808 11:41:18.648000 16726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:41:18.674379 16726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:41:18.678192 16726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:41:18.678422 16726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:41:18.679312 16726 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:41:18.680560 16726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:41:18.680647 16726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:41:18.680738 16726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:41:18.680781 16726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:41:18.710892 16726 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.65:43595
I20260808 11:41:18.710937 16918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.65:43595 every 8 connection(s)
I20260808 11:41:18.712087 16726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
I20260808 11:41:18.719316 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 16726
I20260808 11:41:18.719522 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance
I20260808 11:41:18.720132 24881 external_mini_cluster.cc:1428] Setting key 03b4e6af0a08ef57753a699bc4fe168e
Aug 08 11:41:18 dist-test-slave-tc2s krb5kdc[16567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189278, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:41:18.731649 16924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:18.714475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:49149 (local address 127.24.76.126:33899)
0808 11:41:18.714884 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0808 11:41:18.714889 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:41:18.715726 (+ 837us) server_negotiation.cc:407] Connection header received
0808 11:41:18.717538 (+ 1812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:41:18.717545 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:41:18.717622 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:41:18.717718 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:41:18.719438 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:18.720202 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:18.722074 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:18.722511 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:18.722609 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:18.726399 (+ 3790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:41:18.726404 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:41:18.726407 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:41:18.726436 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:41:18.728757 (+ 2321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:18.729426 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:18.729430 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:18.729432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:18.729571 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:18.730001 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:18.730005 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:18.730007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:18.730264 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:41:18.730398 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 11:41:18.731086 (+ 688us) server_negotiation.cc:299] Negotiation successful
0808 11:41:18.731311 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":216,"threads_started":1}
I20260808 11:41:18.732491 16927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:18.714885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.65:49149)
0808 11:41:18.715567 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:18.715606 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 11:41:18.717274 (+ 1668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:41:18.718021 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:41:18.718032 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:41:18.718716 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:41:18.719264 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:41:18.719282 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:41:18.720370 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:41:18.720377 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:41:18.721921 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:41:18.721944 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:41:18.722741 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:41:18.722750 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:41:18.723035 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:18.723698 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:41:18.723726 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:41:18.726199 (+ 2473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:41:18.728886 (+ 2687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:41:18.728895 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:41:18.728910 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:41:18.729305 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:41:18.729676 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:41:18.729682 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:41:18.729686 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:41:18.729865 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:41:18.730393 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:41:18.730400 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:41:18.730709 (+ 309us) client_negotiation.cc:769] Sending connection context
0808 11:41:18.730928 (+ 219us) client_negotiation.cc:240] Negotiation successful
0808 11:41:18.731396 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":257,"threads_started":1}
I20260808 11:41:18.733922 16920 heartbeater.cc:347] Connected to a master server at 127.24.76.126:33899
I20260808 11:41:18.734234 16920 heartbeater.cc:464] Registering TS with master...
I20260808 11:41:18.734894 16920 heartbeater.cc:511] Master 127.24.76.126:33899 requested a full tablet report, sending...
I20260808 11:41:18.736841 16636 ts_manager.cc:194] Registered new tserver with Master: f07b914ccc874645a5998194df57f1cc (127.24.76.65:43595)
I20260808 11:41:18.738324 16636 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.65@KRBTEST.COM'} at 127.24.76.65:49149
WARNING: no policy specified for kudu/127.24.76.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab.
Entry for principal kudu/127.24.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab.
I20260808 11:41:18.778928 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.66:0
--local_ip_for_outbound_sockets=127.24.76.66
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:33899
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:33281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:41:19.104413 16931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:41:19.104682 16931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:41:19.104815 16931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:41:19.111845 16931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:41:19.111950 16931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:41:19.112099 16931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.66
I20260808 11:41:19.121197 16931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:33281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:33899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.66
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:41:19.122351 16931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:41:19.123550 16931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:41:19.135648 16937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:41:19.135618 16938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:41:19.135573 16941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:41:19.136588 16931 server_base.cc:1034] running on GCE node
I20260808 11:41:19.137156 16931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:41:19.138037 16931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:41:19.139233 16931 hybrid_clock.cc:648] HybridClock initialized: now 1786189279139180 us; error 49 us; skew 500 ppm
I20260808 11:41:19.139523 16931 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:41:19 dist-test-slave-tc2s krb5kdc[16567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189279, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:41:19.142496 16931 init.cc:377] Logged in from keytab as kudu/127.24.76.66@KRBTEST.COM (short username kudu)
I20260808 11:41:19.143815 16931 webserver.cc:460] Webserver started at http://127.24.76.66:33147/ using document root <none> and password file <none>
I20260808 11:41:19.144438 16931 fs_manager.cc:359] Metadata directory not provided
I20260808 11:41:19.144554 16931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:41:19.144832 16931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:41:19.147677 16931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/data/instance:
uuid: "1d706a7ad7154186906a5efb55efd4ca"
format_stamp: "Formatted at 2026-08-08 11:41:19 on dist-test-slave-tc2s"
server_key: "883ccd7817cac0b8b831f0d12399bbe2"
server_key_iv: "3d759670ab5cbd662be1256c3286388c"
server_key_version: "encryptionkey@0"
I20260808 11:41:19.148488 16931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance:
uuid: "1d706a7ad7154186906a5efb55efd4ca"
format_stamp: "Formatted at 2026-08-08 11:41:19 on dist-test-slave-tc2s"
server_key: "883ccd7817cac0b8b831f0d12399bbe2"
server_key_iv: "3d759670ab5cbd662be1256c3286388c"
server_key_version: "encryptionkey@0"
I20260808 11:41:19.153425 16931 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 11:41:19.156708 16949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:41:19.157915 16931 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 11:41:19.158075 16931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/wal
uuid: "1d706a7ad7154186906a5efb55efd4ca"
format_stamp: "Formatted at 2026-08-08 11:41:19 on dist-test-slave-tc2s"
server_key: "883ccd7817cac0b8b831f0d12399bbe2"
server_key_iv: "3d759670ab5cbd662be1256c3286388c"
server_key_version: "encryptionkey@0"
I20260808 11:41:19.158242 16931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:41:19.186221 16931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:41:19.190011 16931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:41:19.190272 16931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:41:19.191195 16931 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:41:19.192463 16931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:41:19.192540 16931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:41:19.192658 16931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:41:19.192721 16931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:41:19.222366 16931 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.66:33379
I20260808 11:41:19.222427 17122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.66:33379 every 8 connection(s)
I20260808 11:41:19.223582 16931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
I20260808 11:41:19.233426 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 16931
I20260808 11:41:19.233614 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance
I20260808 11:41:19.234086 24881 external_mini_cluster.cc:1428] Setting key a216e7523de0ea92921bdafb09b391c8
Aug 08 11:41:19 dist-test-slave-tc2s krb5kdc[16567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189279, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:41:19.242831 16924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:19.225980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:60659 (local address 127.24.76.126:33899)
0808 11:41:19.226155 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 11:41:19.226160 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:41:19.227301 (+ 1141us) server_negotiation.cc:407] Connection header received
0808 11:41:19.229107 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:41:19.229113 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:41:19.229192 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:41:19.229279 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:41:19.230770 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:19.231491 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:19.232990 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:19.233550 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:19.233678 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:19.237221 (+ 3543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:41:19.237226 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:41:19.237229 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:41:19.237261 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:41:19.239880 (+ 2619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:19.240612 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:19.240616 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:19.240619 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:19.240750 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:19.241170 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:19.241173 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:19.241175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:19.241429 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:41:19.241565 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 11:41:19.242279 (+ 714us) server_negotiation.cc:299] Negotiation successful
0808 11:41:19.242514 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 11:41:19.243813 17127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:19.226470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.66:60659)
0808 11:41:19.227148 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:19.227191 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 11:41:19.228840 (+ 1649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:41:19.229513 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:41:19.229521 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:41:19.230051 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:41:19.230599 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:41:19.230618 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:41:19.231638 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:41:19.231642 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:41:19.232852 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:41:19.232864 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:41:19.233759 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:41:19.233767 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:41:19.234033 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:19.234685 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:41:19.234714 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:41:19.237010 (+ 2296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:41:19.240043 (+ 3033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:41:19.240051 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:41:19.240067 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:41:19.240485 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:41:19.240852 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:41:19.240857 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:41:19.240861 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:41:19.241055 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:41:19.241557 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:41:19.241565 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:41:19.241872 (+ 307us) client_negotiation.cc:769] Sending connection context
0808 11:41:19.242150 (+ 278us) client_negotiation.cc:240] Negotiation successful
0808 11:41:19.242624 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":267,"threads_started":1}
I20260808 11:41:19.245174 17123 heartbeater.cc:347] Connected to a master server at 127.24.76.126:33899
I20260808 11:41:19.245455 17123 heartbeater.cc:464] Registering TS with master...
I20260808 11:41:19.246163 17123 heartbeater.cc:511] Master 127.24.76.126:33899 requested a full tablet report, sending...
I20260808 11:41:19.247651 16636 ts_manager.cc:194] Registered new tserver with Master: 1d706a7ad7154186906a5efb55efd4ca (127.24.76.66:33379)
I20260808 11:41:19.248654 16636 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.66@KRBTEST.COM'} at 127.24.76.66:60659
WARNING: no policy specified for kudu/127.24.76.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab.
Entry for principal kudu/127.24.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab.
I20260808 11:41:19.294704 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.67:0
--local_ip_for_outbound_sockets=127.24.76.67
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:33899
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:33281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:41:19.619611 17131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:41:19.619870 17131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:41:19.620007 17131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:41:19.626947 17131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:41:19.627058 17131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:41:19.627223 17131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.67
I20260808 11:41:19.636435 17131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:33281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:33899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.67
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:41:19.637636 17131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:41:19.638949 17131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:41:19.651540 17141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:41:19.651510 17145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:41:19.651476 17142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:41:19.652330 17131 server_base.cc:1034] running on GCE node
I20260808 11:41:19.652918 17131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:41:19.653851 17131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:41:19.655114 17131 hybrid_clock.cc:648] HybridClock initialized: now 1786189279655060 us; error 35 us; skew 500 ppm
I20260808 11:41:19.655411 17131 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:41:19 dist-test-slave-tc2s krb5kdc[16567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189279, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:41:19.658460 17131 init.cc:377] Logged in from keytab as kudu/127.24.76.67@KRBTEST.COM (short username kudu)
I20260808 11:41:19.659740 17131 webserver.cc:460] Webserver started at http://127.24.76.67:44925/ using document root <none> and password file <none>
I20260808 11:41:19.660377 17131 fs_manager.cc:359] Metadata directory not provided
I20260808 11:41:19.660477 17131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:41:19.660740 17131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:41:19.663630 17131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/data/instance:
uuid: "f74fa58be8dd444eb55034cdc2ad57c9"
format_stamp: "Formatted at 2026-08-08 11:41:19 on dist-test-slave-tc2s"
server_key: "f178b819e5f67340e85b946763a65afa"
server_key_iv: "ab1acfed7b9269137915a17a89a21f37"
server_key_version: "encryptionkey@0"
I20260808 11:41:19.664325 17131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance:
uuid: "f74fa58be8dd444eb55034cdc2ad57c9"
format_stamp: "Formatted at 2026-08-08 11:41:19 on dist-test-slave-tc2s"
server_key: "f178b819e5f67340e85b946763a65afa"
server_key_iv: "ab1acfed7b9269137915a17a89a21f37"
server_key_version: "encryptionkey@0"
I20260808 11:41:19.669194 17131 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 11:41:19.672355 17153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:41:19.673558 17131 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 11:41:19.673687 17131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/wal
uuid: "f74fa58be8dd444eb55034cdc2ad57c9"
format_stamp: "Formatted at 2026-08-08 11:41:19 on dist-test-slave-tc2s"
server_key: "f178b819e5f67340e85b946763a65afa"
server_key_iv: "ab1acfed7b9269137915a17a89a21f37"
server_key_version: "encryptionkey@0"
I20260808 11:41:19.673840 17131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:41:19.692385 17131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:41:19.695995 17131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:41:19.696218 17131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:41:19.697082 17131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:41:19.698372 17131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:41:19.698443 17131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:41:19.698541 17131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:41:19.698585 17131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:41:19.728186 17131 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.67:44575
I20260808 11:41:19.728227 17324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.67:44575 every 8 connection(s)
I20260808 11:41:19.729307 17131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
I20260808 11:41:19.738118 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 17131
I20260808 11:41:19.738291 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance
I20260808 11:41:19.738708 24881 external_mini_cluster.cc:1428] Setting key db529233cfdc596ac271be4d498c70d0
I20260808 11:41:19.741071 16920 heartbeater.cc:503] Master 127.24.76.126:33899 was elected leader, sending a full tablet report...
Aug 08 11:41:19 dist-test-slave-tc2s krb5kdc[16567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189279, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:41:19.748060 16924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:19.731625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:57583 (local address 127.24.76.126:33899)
0808 11:41:19.731803 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 11:41:19.731808 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:41:19.732896 (+ 1088us) server_negotiation.cc:407] Connection header received
0808 11:41:19.734727 (+ 1831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:41:19.734733 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:41:19.734821 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:41:19.734906 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:41:19.736309 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:19.737093 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:19.738724 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:19.739216 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:19.739348 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:19.743026 (+ 3678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:41:19.743031 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:41:19.743034 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:41:19.743064 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:41:19.745555 (+ 2491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:19.746237 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:19.746241 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:19.746244 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:19.746370 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:19.746705 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:19.746708 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:19.746709 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:19.746987 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:41:19.747099 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 11:41:19.747651 (+ 552us) server_negotiation.cc:299] Negotiation successful
0808 11:41:19.747805 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 11:41:19.748937 17327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:19.732049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.67:57583)
0808 11:41:19.732737 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:19.732783 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 11:41:19.734467 (+ 1684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:41:19.735137 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:41:19.735145 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:41:19.735629 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:41:19.736146 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:41:19.736163 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:41:19.737242 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:41:19.737246 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:41:19.738588 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:41:19.738601 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:41:19.739437 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:41:19.739446 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:41:19.739713 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:19.740349 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:41:19.740375 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:41:19.742839 (+ 2464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:41:19.745664 (+ 2825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:41:19.745670 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:41:19.745680 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:41:19.746137 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:41:19.746451 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:41:19.746454 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:41:19.746456 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:41:19.746623 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:41:19.747086 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:41:19.747091 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:41:19.747364 (+ 273us) client_negotiation.cc:769] Sending connection context
0808 11:41:19.747571 (+ 207us) client_negotiation.cc:240] Negotiation successful
0808 11:41:19.747909 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":261,"threads_started":1}
I20260808 11:41:19.750207 17325 heartbeater.cc:347] Connected to a master server at 127.24.76.126:33899
I20260808 11:41:19.750483 17325 heartbeater.cc:464] Registering TS with master...
I20260808 11:41:19.751148 17325 heartbeater.cc:511] Master 127.24.76.126:33899 requested a full tablet report, sending...
I20260808 11:41:19.752602 16636 ts_manager.cc:194] Registered new tserver with Master: f74fa58be8dd444eb55034cdc2ad57c9 (127.24.76.67:44575)
I20260808 11:41:19.753566 16636 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.67@KRBTEST.COM'} at 127.24.76.67:57583
I20260808 11:41:19.754130 24881 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:41:19.770257 16924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:19.758424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43046 (local address 127.24.76.126:33899)
0808 11:41:19.758580 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 11:41:19.758584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:41:19.759013 (+ 429us) server_negotiation.cc:407] Connection header received
0808 11:41:19.759181 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:41:19.759186 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:41:19.759241 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:41:19.759315 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:41:19.760657 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:19.761381 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:19.762904 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:19.763303 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:19.763410 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:19.765670 (+ 2260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:41:19.765675 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:41:19.765678 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:41:19.765706 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:41:19.767830 (+ 2124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:19.768420 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:19.768425 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:19.768429 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:19.768556 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:19.768910 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:19.768914 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:19.768916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:19.769205 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:41:19.769441 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0808 11:41:19.769754 (+ 313us) server_negotiation.cc:299] Negotiation successful
0808 11:41:19.769979 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 11:41:19.773790 16636 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43046:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:41:19.780705 16636 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:41:19.803144 17348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:19.791809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53054 (local address 127.24.76.67:44575)
0808 11:41:19.793136 (+ 1327us) server_negotiation.cc:206] Beginning negotiation
0808 11:41:19.793142 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:41:19.793698 (+ 556us) server_negotiation.cc:407] Connection header received
0808 11:41:19.794011 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:41:19.794021 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:41:19.794281 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:41:19.794475 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:41:19.795436 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:19.797174 (+ 1738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:19.800042 (+ 2868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:19.801572 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:19.801678 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:19.801955 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0808 11:41:19.802646 (+ 691us) server_negotiation.cc:299] Negotiation successful
0808 11:41:19.802811 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1188,"thread_start_us":1041,"threads_started":1}
I20260808 11:41:19.804277 17347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:19.791656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:44575 (local address 127.0.0.1:53054)
0808 11:41:19.793556 (+ 1900us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:19.793594 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 11:41:19.793756 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:41:19.794776 (+ 1020us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:41:19.794783 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:41:19.794812 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:41:19.795216 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:41:19.795230 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:41:19.797374 (+ 2144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:41:19.797381 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:41:19.799824 (+ 2443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:41:19.799841 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:41:19.802151 (+ 2310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:41:19.802158 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:41:19.802380 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:19.802430 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 11:41:19.803638 (+ 1208us) client_negotiation.cc:240] Negotiation successful
0808 11:41:19.803870 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1710,"thread_start_us":1607,"threads_started":1}
I20260808 11:41:19.812671 17225 tablet_service.cc:1467] Processing CreateTablet for tablet 278c2e4cc5a542fdb195c33f26a76fac (DEFAULT_TABLE table=test-table [id=fd96ba8bf8624ba591fbc7a032f200cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:41:19.814448 17349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:19.794104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:43595 (local address 127.0.0.1:36836)
0808 11:41:19.806348 (+ 12244us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:19.806365 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:41:19.806471 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:41:19.807950 (+ 1479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:41:19.807956 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:41:19.807997 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:41:19.808410 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:41:19.808420 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:41:19.810275 (+ 1855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:41:19.810281 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:41:19.812001 (+ 1720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:41:19.812013 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:41:19.813611 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:41:19.813618 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:41:19.813792 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:19.813823 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 11:41:19.813948 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 11:41:19.814090 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12132,"thread_start_us":14636,"threads_started":1}
I20260808 11:41:19.814447 17350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:19.792457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41602 (local address 127.24.76.66:33379)
0808 11:41:19.806686 (+ 14229us) server_negotiation.cc:206] Beginning negotiation
0808 11:41:19.806692 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:41:19.806715 (+ 23us) server_negotiation.cc:407] Connection header received
0808 11:41:19.806794 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:41:19.806800 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:41:19.806981 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:41:19.807152 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:41:19.807856 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:19.809351 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:19.811491 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:19.812939 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:19.813681 (+ 742us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:19.813863 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 11:41:19.813967 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 11:41:19.814126 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14092,"thread_start_us":13942,"threads_started":1}
I20260808 11:41:19.814534 17347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:19.792209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:33379 (local address 127.0.0.1:41602)
0808 11:41:19.804702 (+ 12493us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:19.804723 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 11:41:19.804827 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:41:19.807153 (+ 2326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:41:19.807158 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:41:19.807183 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:41:19.807350 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:41:19.807362 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:41:19.809506 (+ 2144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:41:19.809511 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:41:19.811323 (+ 1812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:41:19.811339 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:41:19.813139 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:41:19.813146 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:41:19.813315 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:19.813354 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 11:41:19.813424 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 11:41:19.814249 (+ 825us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12388,"thread_start_us":13900,"threads_started":1}
I20260808 11:41:19.814846 17225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 278c2e4cc5a542fdb195c33f26a76fac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:41:19.815207 17353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:19.794762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36836 (local address 127.24.76.65:43595)
0808 11:41:19.807120 (+ 12358us) server_negotiation.cc:206] Beginning negotiation
0808 11:41:19.807129 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:41:19.807161 (+ 32us) server_negotiation.cc:407] Connection header received
0808 11:41:19.807264 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:41:19.807272 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:41:19.807505 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:41:19.807700 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:41:19.808685 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:19.810114 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:19.812152 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:19.813160 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:19.813301 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:19.813563 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0808 11:41:19.814708 (+ 1145us) server_negotiation.cc:299] Negotiation successful
0808 11:41:19.814877 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12173,"thread_start_us":12028,"threads_started":1}
I20260808 11:41:19.818135 16822 tablet_service.cc:1467] Processing CreateTablet for tablet 278c2e4cc5a542fdb195c33f26a76fac (DEFAULT_TABLE table=test-table [id=fd96ba8bf8624ba591fbc7a032f200cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:41:19.818356 17018 tablet_service.cc:1467] Processing CreateTablet for tablet 278c2e4cc5a542fdb195c33f26a76fac (DEFAULT_TABLE table=test-table [id=fd96ba8bf8624ba591fbc7a032f200cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:41:19.819401 16822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 278c2e4cc5a542fdb195c33f26a76fac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:41:19.819681 17018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 278c2e4cc5a542fdb195c33f26a76fac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:41:19.830819 17355 tablet_bootstrap.cc:492] T 278c2e4cc5a542fdb195c33f26a76fac P f07b914ccc874645a5998194df57f1cc: Bootstrap starting.
I20260808 11:41:19.831017 17354 tablet_bootstrap.cc:492] T 278c2e4cc5a542fdb195c33f26a76fac P f74fa58be8dd444eb55034cdc2ad57c9: Bootstrap starting.
I20260808 11:41:19.833936 17355 tablet_bootstrap.cc:654] T 278c2e4cc5a542fdb195c33f26a76fac P f07b914ccc874645a5998194df57f1cc: Neither blocks nor log segments found. Creating new log.
I20260808 11:41:19.833988 17357 tablet_bootstrap.cc:492] T 278c2e4cc5a542fdb195c33f26a76fac P 1d706a7ad7154186906a5efb55efd4ca: Bootstrap starting.
I20260808 11:41:19.834461 17354 tablet_bootstrap.cc:654] T 278c2e4cc5a542fdb195c33f26a76fac P f74fa58be8dd444eb55034cdc2ad57c9: Neither blocks nor log segments found. Creating new log.
I20260808 11:41:19.835320 17355 log.cc:824] T 278c2e4cc5a542fdb195c33f26a76fac P f07b914ccc874645a5998194df57f1cc: Log is configured to *not* fsync() on all Append() calls
I20260808 11:41:19.835459 17354 log.cc:824] T 278c2e4cc5a542fdb195c33f26a76fac P f74fa58be8dd444eb55034cdc2ad57c9: Log is configured to *not* fsync() on all Append() calls
I20260808 11:41:19.837471 17357 tablet_bootstrap.cc:654] T 278c2e4cc5a542fdb195c33f26a76fac P 1d706a7ad7154186906a5efb55efd4ca: Neither blocks nor log segments found. Creating new log.
I20260808 11:41:19.838420 17357 log.cc:824] T 278c2e4cc5a542fdb195c33f26a76fac P 1d706a7ad7154186906a5efb55efd4ca: Log is configured to *not* fsync() on all Append() calls
I20260808 11:41:19.838475 17355 tablet_bootstrap.cc:492] T 278c2e4cc5a542fdb195c33f26a76fac P f07b914ccc874645a5998194df57f1cc: No bootstrap required, opened a new log
I20260808 11:41:19.838575 17354 tablet_bootstrap.cc:492] T 278c2e4cc5a542fdb195c33f26a76fac P f74fa58be8dd444eb55034cdc2ad57c9: No bootstrap required, opened a new log
I20260808 11:41:19.838745 17355 ts_tablet_manager.cc:1397] T 278c2e4cc5a542fdb195c33f26a76fac P f07b914ccc874645a5998194df57f1cc: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260808 11:41:19.838798 17354 ts_tablet_manager.cc:1397] T 278c2e4cc5a542fdb195c33f26a76fac P f74fa58be8dd444eb55034cdc2ad57c9: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260808 11:41:19.841352 17357 tablet_bootstrap.cc:492] T 278c2e4cc5a542fdb195c33f26a76fac P 1d706a7ad7154186906a5efb55efd4ca: No bootstrap required, opened a new log
I20260808 11:41:19.841594 17357 ts_tablet_manager.cc:1397] T 278c2e4cc5a542fdb195c33f26a76fac P 1d706a7ad7154186906a5efb55efd4ca: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260808 11:41:19.849309 17354 raft_consensus.cc:348] T 278c2e4cc5a542fdb195c33f26a76fac P f74fa58be8dd444eb55034cdc2ad57c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74fa58be8dd444eb55034cdc2ad57c9" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44575 } } peers { permanent_uuid: "1d706a7ad7154186906a5efb55efd4ca" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33379 } } peers { permanent_uuid: "f07b914ccc874645a5998194df57f1cc" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43595 } }
I20260808 11:41:19.849174 17355 raft_consensus.cc:348] T 278c2e4cc5a542fdb195c33f26a76fac P f07b914ccc874645a5998194df57f1cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74fa58be8dd444eb55034cdc2ad57c9" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44575 } } peers { permanent_uuid: "1d706a7ad7154186906a5efb55efd4ca" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33379 } } peers { permanent_uuid: "f07b914ccc874645a5998194df57f1cc" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43595 } }
I20260808 11:41:19.849710 17355 raft_consensus.cc:374] T 278c2e4cc5a542fdb195c33f26a76fac P f07b914ccc874645a5998194df57f1cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:41:19.849709 17354 raft_consensus.cc:374] T 278c2e4cc5a542fdb195c33f26a76fac P f74fa58be8dd444eb55034cdc2ad57c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:41:19.849810 17354 raft_consensus.cc:729] T 278c2e4cc5a542fdb195c33f26a76fac P f74fa58be8dd444eb55034cdc2ad57c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f74fa58be8dd444eb55034cdc2ad57c9, State: Initialized, Role: FOLLOWER
I20260808 11:41:19.849858 17355 raft_consensus.cc:729] T 278c2e4cc5a542fdb195c33f26a76fac P f07b914ccc874645a5998194df57f1cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f07b914ccc874645a5998194df57f1cc, State: Initialized, Role: FOLLOWER
I20260808 11:41:19.850498 17357 raft_consensus.cc:348] T 278c2e4cc5a542fdb195c33f26a76fac P 1d706a7ad7154186906a5efb55efd4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74fa58be8dd444eb55034cdc2ad57c9" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44575 } } peers { permanent_uuid: "1d706a7ad7154186906a5efb55efd4ca" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33379 } } peers { permanent_uuid: "f07b914ccc874645a5998194df57f1cc" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43595 } }
I20260808 11:41:19.850495 17354 consensus_queue.cc:260] T 278c2e4cc5a542fdb195c33f26a76fac P f74fa58be8dd444eb55034cdc2ad57c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74fa58be8dd444eb55034cdc2ad57c9" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44575 } } peers { permanent_uuid: "1d706a7ad7154186906a5efb55efd4ca" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33379 } } peers { permanent_uuid: "f07b914ccc874645a5998194df57f1cc" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43595 } }
I20260808 11:41:19.850520 17355 consensus_queue.cc:260] T 278c2e4cc5a542fdb195c33f26a76fac P f07b914ccc874645a5998194df57f1cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74fa58be8dd444eb55034cdc2ad57c9" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44575 } } peers { permanent_uuid: "1d706a7ad7154186906a5efb55efd4ca" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33379 } } peers { permanent_uuid: "f07b914ccc874645a5998194df57f1cc" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43595 } }
I20260808 11:41:19.850807 17357 raft_consensus.cc:374] T 278c2e4cc5a542fdb195c33f26a76fac P 1d706a7ad7154186906a5efb55efd4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:41:19.850927 17357 raft_consensus.cc:729] T 278c2e4cc5a542fdb195c33f26a76fac P 1d706a7ad7154186906a5efb55efd4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d706a7ad7154186906a5efb55efd4ca, State: Initialized, Role: FOLLOWER
I20260808 11:41:19.851611 17357 consensus_queue.cc:260] T 278c2e4cc5a542fdb195c33f26a76fac P 1d706a7ad7154186906a5efb55efd4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74fa58be8dd444eb55034cdc2ad57c9" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44575 } } peers { permanent_uuid: "1d706a7ad7154186906a5efb55efd4ca" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33379 } } peers { permanent_uuid: "f07b914ccc874645a5998194df57f1cc" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43595 } }
I20260808 11:41:19.852101 17325 heartbeater.cc:503] Master 127.24.76.126:33899 was elected leader, sending a full tablet report...
I20260808 11:41:19.852591 17354 ts_tablet_manager.cc:1428] T 278c2e4cc5a542fdb195c33f26a76fac P f74fa58be8dd444eb55034cdc2ad57c9: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260808 11:41:19.853437 17123 heartbeater.cc:503] Master 127.24.76.126:33899 was elected leader, sending a full tablet report...
I20260808 11:41:19.854132 17357 ts_tablet_manager.cc:1428] T 278c2e4cc5a542fdb195c33f26a76fac P 1d706a7ad7154186906a5efb55efd4ca: Time spent starting tablet: real 0.012s user 0.014s sys 0.000s
I20260808 11:41:19.854868 17355 ts_tablet_manager.cc:1428] T 278c2e4cc5a542fdb195c33f26a76fac P f07b914ccc874645a5998194df57f1cc: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
W20260808 11:41:19.964279 16922 tablet.cc:2369] T 278c2e4cc5a542fdb195c33f26a76fac P f07b914ccc874645a5998194df57f1cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:41:19.975358 17124 tablet.cc:2369] T 278c2e4cc5a542fdb195c33f26a76fac P 1d706a7ad7154186906a5efb55efd4ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:41:19.980752 17326 tablet.cc:2369] T 278c2e4cc5a542fdb195c33f26a76fac P f74fa58be8dd444eb55034cdc2ad57c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:41:20.050239 17364 raft_consensus.cc:482] T 278c2e4cc5a542fdb195c33f26a76fac P f07b914ccc874645a5998194df57f1cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:41:20.050551 17364 raft_consensus.cc:504] T 278c2e4cc5a542fdb195c33f26a76fac P f07b914ccc874645a5998194df57f1cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74fa58be8dd444eb55034cdc2ad57c9" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44575 } } peers { permanent_uuid: "1d706a7ad7154186906a5efb55efd4ca" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33379 } } peers { permanent_uuid: "f07b914ccc874645a5998194df57f1cc" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43595 } }
I20260808 11:41:20.052353 17364 leader_election.cc:288] T 278c2e4cc5a542fdb195c33f26a76fac P f07b914ccc874645a5998194df57f1cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f74fa58be8dd444eb55034cdc2ad57c9 (127.24.76.67:44575), 1d706a7ad7154186906a5efb55efd4ca (127.24.76.66:33379)
I20260808 11:41:20.059089 17348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:20.052759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:57313 (local address 127.24.76.67:44575)
0808 11:41:20.053003 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0808 11:41:20.053010 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:41:20.053699 (+ 689us) server_negotiation.cc:407] Connection header received
0808 11:41:20.053916 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:41:20.053923 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:41:20.053992 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:41:20.054071 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:41:20.054643 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:20.055456 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:20.057399 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:20.058202 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:20.058338 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:20.058414 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 11:41:20.058723 (+ 309us) server_negotiation.cc:299] Negotiation successful
0808 11:41:20.058848 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 11:41:20.059135 17350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:20.053011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:59189 (local address 127.24.76.66:33379)
0808 11:41:20.053190 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 11:41:20.053195 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:41:20.053802 (+ 607us) server_negotiation.cc:407] Connection header received
0808 11:41:20.053993 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:41:20.053999 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:41:20.054063 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:41:20.054136 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:41:20.054797 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:20.055537 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:20.057394 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:20.058231 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:20.058379 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:20.058448 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 11:41:20.058744 (+ 296us) server_negotiation.cc:299] Negotiation successful
0808 11:41:20.058879 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 11:41:20.059242 17369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:20.052594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:33379 (local address 127.24.76.65:59189)
0808 11:41:20.053705 (+ 1111us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:20.053723 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 11:41:20.053853 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:41:20.054161 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:41:20.054165 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:41:20.054193 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:41:20.054652 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:41:20.054665 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:41:20.055671 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:41:20.055674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:41:20.057252 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:41:20.057265 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:41:20.058460 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:41:20.058467 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:41:20.058599 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:20.058621 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 11:41:20.058706 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 11:41:20.058825 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":974,"mutex_wait_us":49,"thread_start_us":635,"threads_started":1}
I20260808 11:41:20.059526 17367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:20.052588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:44575 (local address 127.24.76.65:57313)
0808 11:41:20.053585 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:20.053604 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:41:20.053745 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:41:20.054101 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:41:20.054105 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:41:20.054129 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:41:20.054491 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:41:20.054502 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:41:20.055583 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:41:20.055589 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:41:20.057254 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:41:20.057267 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:41:20.058413 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:41:20.058420 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:41:20.058577 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:20.058602 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 11:41:20.058675 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 11:41:20.058791 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":876,"mutex_wait_us":2,"thread_start_us":638,"threads_started":1}
I20260808 11:41:20.060245 17055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "278c2e4cc5a542fdb195c33f26a76fac" candidate_uuid: "f07b914ccc874645a5998194df57f1cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d706a7ad7154186906a5efb55efd4ca" is_pre_election: true
I20260808 11:41:20.060429 17253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "278c2e4cc5a542fdb195c33f26a76fac" candidate_uuid: "f07b914ccc874645a5998194df57f1cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f74fa58be8dd444eb55034cdc2ad57c9" is_pre_election: true
I20260808 11:41:20.060755 17055 raft_consensus.cc:2454] T 278c2e4cc5a542fdb195c33f26a76fac P 1d706a7ad7154186906a5efb55efd4ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f07b914ccc874645a5998194df57f1cc in term 0.
I20260808 11:41:20.060899 17253 raft_consensus.cc:2454] T 278c2e4cc5a542fdb195c33f26a76fac P f74fa58be8dd444eb55034cdc2ad57c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f07b914ccc874645a5998194df57f1cc in term 0.
I20260808 11:41:20.061431 16759 leader_election.cc:302] T 278c2e4cc5a542fdb195c33f26a76fac P f07b914ccc874645a5998194df57f1cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d706a7ad7154186906a5efb55efd4ca, f07b914ccc874645a5998194df57f1cc; no voters:
I20260808 11:41:20.061826 17364 raft_consensus.cc:2781] T 278c2e4cc5a542fdb195c33f26a76fac P f07b914ccc874645a5998194df57f1cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:41:20.062018 17364 raft_consensus.cc:482] T 278c2e4cc5a542fdb195c33f26a76fac P f07b914ccc874645a5998194df57f1cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:41:20.062145 17364 raft_consensus.cc:3037] T 278c2e4cc5a542fdb195c33f26a76fac P f07b914ccc874645a5998194df57f1cc [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:41:20.064276 17364 raft_consensus.cc:504] T 278c2e4cc5a542fdb195c33f26a76fac P f07b914ccc874645a5998194df57f1cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74fa58be8dd444eb55034cdc2ad57c9" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44575 } } peers { permanent_uuid: "1d706a7ad7154186906a5efb55efd4ca" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33379 } } peers { permanent_uuid: "f07b914ccc874645a5998194df57f1cc" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43595 } }
I20260808 11:41:20.064805 17364 leader_election.cc:288] T 278c2e4cc5a542fdb195c33f26a76fac P f07b914ccc874645a5998194df57f1cc [CANDIDATE]: Term 1 election: Requested vote from peers f74fa58be8dd444eb55034cdc2ad57c9 (127.24.76.67:44575), 1d706a7ad7154186906a5efb55efd4ca (127.24.76.66:33379)
I20260808 11:41:20.065186 17253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "278c2e4cc5a542fdb195c33f26a76fac" candidate_uuid: "f07b914ccc874645a5998194df57f1cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f74fa58be8dd444eb55034cdc2ad57c9"
I20260808 11:41:20.065256 17055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "278c2e4cc5a542fdb195c33f26a76fac" candidate_uuid: "f07b914ccc874645a5998194df57f1cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d706a7ad7154186906a5efb55efd4ca"
I20260808 11:41:20.065339 17253 raft_consensus.cc:3037] T 278c2e4cc5a542fdb195c33f26a76fac P f74fa58be8dd444eb55034cdc2ad57c9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:41:20.065457 17055 raft_consensus.cc:3037] T 278c2e4cc5a542fdb195c33f26a76fac P 1d706a7ad7154186906a5efb55efd4ca [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:41:20.067513 17253 raft_consensus.cc:2454] T 278c2e4cc5a542fdb195c33f26a76fac P f74fa58be8dd444eb55034cdc2ad57c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f07b914ccc874645a5998194df57f1cc in term 1.
I20260808 11:41:20.067618 17055 raft_consensus.cc:2454] T 278c2e4cc5a542fdb195c33f26a76fac P 1d706a7ad7154186906a5efb55efd4ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f07b914ccc874645a5998194df57f1cc in term 1.
I20260808 11:41:20.067873 16755 leader_election.cc:302] T 278c2e4cc5a542fdb195c33f26a76fac P f07b914ccc874645a5998194df57f1cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f07b914ccc874645a5998194df57f1cc, f74fa58be8dd444eb55034cdc2ad57c9; no voters:
I20260808 11:41:20.068192 17364 raft_consensus.cc:2781] T 278c2e4cc5a542fdb195c33f26a76fac P f07b914ccc874645a5998194df57f1cc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:41:20.068620 17364 raft_consensus.cc:686] T 278c2e4cc5a542fdb195c33f26a76fac P f07b914ccc874645a5998194df57f1cc [term 1 LEADER]: Becoming Leader. State: Replica: f07b914ccc874645a5998194df57f1cc, State: Running, Role: LEADER
I20260808 11:41:20.069060 17364 consensus_queue.cc:237] T 278c2e4cc5a542fdb195c33f26a76fac P f07b914ccc874645a5998194df57f1cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74fa58be8dd444eb55034cdc2ad57c9" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44575 } } peers { permanent_uuid: "1d706a7ad7154186906a5efb55efd4ca" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33379 } } peers { permanent_uuid: "f07b914ccc874645a5998194df57f1cc" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43595 } }
I20260808 11:41:20.073627 16636 catalog_manager.cc:5495] T 278c2e4cc5a542fdb195c33f26a76fac P f07b914ccc874645a5998194df57f1cc reported cstate change: term changed from 0 to 1, leader changed from <none> to f07b914ccc874645a5998194df57f1cc (127.24.76.65). New cstate: current_term: 1 leader_uuid: "f07b914ccc874645a5998194df57f1cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74fa58be8dd444eb55034cdc2ad57c9" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d706a7ad7154186906a5efb55efd4ca" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f07b914ccc874645a5998194df57f1cc" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43595 } health_report { overall_health: HEALTHY } } }
I20260808 11:41:20.095077 17353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:20.090356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36838 (local address 127.24.76.65:43595)
0808 11:41:20.090526 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 11:41:20.090531 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:41:20.090583 (+ 52us) server_negotiation.cc:407] Connection header received
0808 11:41:20.090752 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:41:20.090756 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:41:20.090833 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:41:20.090948 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:41:20.091233 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:20.091926 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:20.093268 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:20.093655 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:20.093736 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:20.094171 (+ 435us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:41:20.094673 (+ 502us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:41:20.094773 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 11:41:20.094832 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 11:41:20.094894 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 11:41:20.102538 17055 raft_consensus.cc:1261] T 278c2e4cc5a542fdb195c33f26a76fac P 1d706a7ad7154186906a5efb55efd4ca [term 1 FOLLOWER]: Refusing update from remote peer f07b914ccc874645a5998194df57f1cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:41:20.102537 17253 raft_consensus.cc:1261] T 278c2e4cc5a542fdb195c33f26a76fac P f74fa58be8dd444eb55034cdc2ad57c9 [term 1 FOLLOWER]: Refusing update from remote peer f07b914ccc874645a5998194df57f1cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:41:20.103413 17364 consensus_queue.cc:1035] T 278c2e4cc5a542fdb195c33f26a76fac P f07b914ccc874645a5998194df57f1cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d706a7ad7154186906a5efb55efd4ca" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:41:20.103734 17370 consensus_queue.cc:1035] T 278c2e4cc5a542fdb195c33f26a76fac P f07b914ccc874645a5998194df57f1cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f74fa58be8dd444eb55034cdc2ad57c9" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:41:20.122026 17375 mvcc.cc:205] Tried to move back new op lower bound from 7316231291291111424 to 7316231291166572544. Current Snapshot: MvccSnapshot[applied={T|T < 7316231291291111424 or (T in {7316231291291111424})}]
I20260808 11:41:20.137964 16636 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43046:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:41:20.138240 16636 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43046:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:41:20.139892 17376 mvcc.cc:205] Tried to move back new op lower bound from 7316231291291111424 to 7316231291166572544. Current Snapshot: MvccSnapshot[applied={T|T < 7316231291291111424}]
I20260808 11:41:20.143496 16636 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6: Sending DeleteTablet for 3 replicas of tablet 278c2e4cc5a542fdb195c33f26a76fac
I20260808 11:41:20.144469 17225 tablet_service.cc:1514] Processing DeleteTablet for tablet 278c2e4cc5a542fdb195c33f26a76fac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:41:20 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:53054
I20260808 11:41:20.146265 16822 tablet_service.cc:1514] Processing DeleteTablet for tablet 278c2e4cc5a542fdb195c33f26a76fac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:41:20 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:36836
I20260808 11:41:20.147965 17018 tablet_service.cc:1514] Processing DeleteTablet for tablet 278c2e4cc5a542fdb195c33f26a76fac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:41:20 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:41602
I20260808 11:41:20.148095 17389 tablet_replica.cc:330] T 278c2e4cc5a542fdb195c33f26a76fac P f07b914ccc874645a5998194df57f1cc: stopping tablet replica
I20260808 11:41:20.148521 17389 raft_consensus.cc:2229] T 278c2e4cc5a542fdb195c33f26a76fac P f07b914ccc874645a5998194df57f1cc [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:41:20.149071 17387 tablet_replica.cc:330] T 278c2e4cc5a542fdb195c33f26a76fac P f74fa58be8dd444eb55034cdc2ad57c9: stopping tablet replica
I20260808 11:41:20.149068 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 16726
I20260808 11:41:20.150571 17389 raft_consensus.cc:2258] T 278c2e4cc5a542fdb195c33f26a76fac P f07b914ccc874645a5998194df57f1cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:41:20.151181 17387 raft_consensus.cc:2229] T 278c2e4cc5a542fdb195c33f26a76fac P f74fa58be8dd444eb55034cdc2ad57c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:41:20.150703 16821 tablet_service.cc:1514] Processing DeleteTablet for tablet 278c2e4cc5a542fdb195c33f26a76fac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:41:20 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:36836
I20260808 11:41:20.153059 17389 ts_tablet_manager.cc:1905] T 278c2e4cc5a542fdb195c33f26a76fac P f07b914ccc874645a5998194df57f1cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:41:20.155105 17387 raft_consensus.cc:2258] T 278c2e4cc5a542fdb195c33f26a76fac P f74fa58be8dd444eb55034cdc2ad57c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:41:20.159320 17387 ts_tablet_manager.cc:1905] T 278c2e4cc5a542fdb195c33f26a76fac P f74fa58be8dd444eb55034cdc2ad57c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:41:20.160112 17389 ts_tablet_manager.cc:1918] T 278c2e4cc5a542fdb195c33f26a76fac P f07b914ccc874645a5998194df57f1cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:41:20.160336 17389 log.cc:1196] T 278c2e4cc5a542fdb195c33f26a76fac P f07b914ccc874645a5998194df57f1cc: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/wal/wals/278c2e4cc5a542fdb195c33f26a76fac
I20260808 11:41:20.160923 17389 ts_tablet_manager.cc:1939] T 278c2e4cc5a542fdb195c33f26a76fac P f07b914ccc874645a5998194df57f1cc: Deleting consensus metadata
I20260808 11:41:20.162674 16618 catalog_manager.cc:4841] TS f07b914ccc874645a5998194df57f1cc (127.24.76.65:43595): tablet 278c2e4cc5a542fdb195c33f26a76fac (table test-table [id=fd96ba8bf8624ba591fbc7a032f200cf]) successfully deleted
W20260808 11:41:20.163231 16618 catalog_manager.cc:4808] TS f07b914ccc874645a5998194df57f1cc (127.24.76.65:43595): delete failed for tablet 278c2e4cc5a542fdb195c33f26a76fac because the tablet was not found. No further retry: Not found: Tablet not found: 278c2e4cc5a542fdb195c33f26a76fac
I20260808 11:41:20.164103 17390 tablet_replica.cc:330] T 278c2e4cc5a542fdb195c33f26a76fac P 1d706a7ad7154186906a5efb55efd4ca: stopping tablet replica
I20260808 11:41:20.164515 17390 raft_consensus.cc:2229] T 278c2e4cc5a542fdb195c33f26a76fac P 1d706a7ad7154186906a5efb55efd4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:41:20.165001 17390 raft_consensus.cc:2258] T 278c2e4cc5a542fdb195c33f26a76fac P 1d706a7ad7154186906a5efb55efd4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:41:20.166507 17387 ts_tablet_manager.cc:1918] T 278c2e4cc5a542fdb195c33f26a76fac P f74fa58be8dd444eb55034cdc2ad57c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:41:20.166705 17387 log.cc:1196] T 278c2e4cc5a542fdb195c33f26a76fac P f74fa58be8dd444eb55034cdc2ad57c9: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/wal/wals/278c2e4cc5a542fdb195c33f26a76fac
I20260808 11:41:20.166852 17390 ts_tablet_manager.cc:1905] T 278c2e4cc5a542fdb195c33f26a76fac P 1d706a7ad7154186906a5efb55efd4ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:41:20.167289 17387 ts_tablet_manager.cc:1939] T 278c2e4cc5a542fdb195c33f26a76fac P f74fa58be8dd444eb55034cdc2ad57c9: Deleting consensus metadata
I20260808 11:41:20.168937 16618 catalog_manager.cc:4841] TS f74fa58be8dd444eb55034cdc2ad57c9 (127.24.76.67:44575): tablet 278c2e4cc5a542fdb195c33f26a76fac (table test-table [id=fd96ba8bf8624ba591fbc7a032f200cf]) successfully deleted
I20260808 11:41:20.172693 17390 ts_tablet_manager.cc:1918] T 278c2e4cc5a542fdb195c33f26a76fac P 1d706a7ad7154186906a5efb55efd4ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:41:20.172824 17390 log.cc:1196] T 278c2e4cc5a542fdb195c33f26a76fac P 1d706a7ad7154186906a5efb55efd4ca: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/wal/wals/278c2e4cc5a542fdb195c33f26a76fac
I20260808 11:41:20.173341 17390 ts_tablet_manager.cc:1939] T 278c2e4cc5a542fdb195c33f26a76fac P 1d706a7ad7154186906a5efb55efd4ca: Deleting consensus metadata
Aug 08 11:41:20 dist-test-slave-tc2s krb5kdc[16567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.65@KRBTEST.COM
I20260808 11:41:20.174964 16622 catalog_manager.cc:4841] TS 1d706a7ad7154186906a5efb55efd4ca (127.24.76.66:33379): tablet 278c2e4cc5a542fdb195c33f26a76fac (table test-table [id=fd96ba8bf8624ba591fbc7a032f200cf]) successfully deleted
I20260808 11:41:20.182641 17353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:20.149933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36854 (local address 127.24.76.65:43595)
0808 11:41:20.150109 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 11:41:20.150116 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:41:20.163455 (+ 13339us) server_negotiation.cc:407] Connection header received
0808 11:41:20.163662 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:41:20.163668 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:41:20.163747 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:41:20.163837 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:41:20.166001 (+ 2164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:20.167221 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:20.169321 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:20.170108 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:20.170243 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:20.175580 (+ 5337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:41:20.175596 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:41:20.175619 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:41:20.175665 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:41:20.179683 (+ 4018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:20.180438 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:20.180455 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:20.180459 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:20.180639 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:20.180980 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:20.180985 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:20.180988 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:20.181458 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:41:20.181752 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0808 11:41:20.182019 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 11:41:20.182256 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 11:41:20.183287 16913 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:41:20.293581 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 16726
I20260808 11:41:20.318668 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 16931
Aug 08 11:41:20 dist-test-slave-tc2s krb5kdc[16567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.66@KRBTEST.COM
I20260808 11:41:20.333540 17350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:20.319419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41616 (local address 127.24.76.66:33379)
0808 11:41:20.319606 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 11:41:20.319612 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:41:20.319633 (+ 21us) server_negotiation.cc:407] Connection header received
0808 11:41:20.319855 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:41:20.319859 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:41:20.319926 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:41:20.320024 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:41:20.321409 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:20.322208 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:20.323516 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:20.323902 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:20.324013 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:20.328197 (+ 4184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:41:20.328211 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:41:20.328226 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:41:20.328259 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:41:20.330864 (+ 2605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:20.331496 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:20.331503 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:20.331505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:20.331630 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:20.331994 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:20.331999 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:20.332002 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:20.332408 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:41:20.332737 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0808 11:41:20.332994 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 11:41:20.333204 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 11:41:20.334408 17110 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:41:20.422623 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 16931
I20260808 11:41:20.450507 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 17131
Aug 08 11:41:20 dist-test-slave-tc2s krb5kdc[16567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.67@KRBTEST.COM
I20260808 11:41:20.464977 17348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:20.451206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53070 (local address 127.24.76.67:44575)
0808 11:41:20.451357 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 11:41:20.451361 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:41:20.451380 (+ 19us) server_negotiation.cc:407] Connection header received
0808 11:41:20.451539 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:41:20.451543 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:41:20.451616 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:41:20.451699 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:41:20.452982 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:20.453761 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:20.455104 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:20.455520 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:20.455614 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:20.460013 (+ 4399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:41:20.460029 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:41:20.460047 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:41:20.460080 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:41:20.462789 (+ 2709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:20.463370 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:20.463377 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:20.463379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:20.463515 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:20.463804 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:20.463807 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:20.463809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:20.464155 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:41:20.464368 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0808 11:41:20.464568 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 11:41:20.464719 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 11:41:20.465595 17319 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:41:20.552024 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 17131
I20260808 11:41:20.572868 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 16590
I20260808 11:41:20.574129 16674 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:41:20.682833 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 16590
I20260808 11:41:40.710111 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:33899
--webserver_interface=127.24.76.126
--webserver_port=36373
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:33281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:33899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:41:41.054651 17397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:41:41.054929 17397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:41:41.055056 17397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:41:41.062052 17397 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 11:41:41.062150 17397 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 11:41:41.062218 17397 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 11:41:41.062285 17397 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:41:41.062364 17397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:41:41.062460 17397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:41:41.062523 17397 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:41:41.062605 17397 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:41:41.072093 17397 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:33281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:33899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:33899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=36373
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:41:41.073335 17397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:41:41.074651 17397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:41:41.098664 17408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:41:41.098729 17407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:41:41.098707 17410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:41:41.099568 17397 server_base.cc:1034] running on GCE node
I20260808 11:41:41.100558 17397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:41:41.102058 17397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:41:41.103276 17397 hybrid_clock.cc:648] HybridClock initialized: now 1786189301103210 us; error 33 us; skew 500 ppm
I20260808 11:41:41.103663 17397 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:41:41 dist-test-slave-tc2s krb5kdc[16567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189301, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:41:41.107215 17397 init.cc:377] Logged in from keytab as kudu/127.24.76.126@KRBTEST.COM (short username kudu)
I20260808 11:41:41.108839 17397 webserver.cc:460] Webserver started at http://127.24.76.126:36373/ using document root <none> and password file <none>
I20260808 11:41:41.109579 17397 fs_manager.cc:359] Metadata directory not provided
I20260808 11:41:41.109691 17397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:41:41.115516 17397 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 11:41:41.118326 17418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:41:41.119628 17397 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260808 11:41:41.119809 17397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "861318038ac14ba6acca16718dbd7fb6"
format_stamp: "Formatted at 2026-08-08 11:41:18 on dist-test-slave-tc2s"
server_key: "f5a9b17562f9d7e52d51fed9058ecc97"
server_key_iv: "d5caf735e621b5375e1ba3cf8385355d"
server_key_version: "encryptionkey@0"
I20260808 11:41:41.120823 17397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:41:41.141100 17397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:41:41.144938 17397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:41:41.145213 17397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:41:41.163152 17397 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:33899
I20260808 11:41:41.163182 17497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:33899 every 8 connection(s)
I20260808 11:41:41.164628 17397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:41:41.171494 17498 sys_catalog.cc:263] Verifying existing consensus state
I20260808 11:41:41.171641 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 17397
I20260808 11:41:41.172657 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.65:43595
--local_ip_for_outbound_sockets=127.24.76.65
--tserver_master_addrs=127.24.76.126:33899
--webserver_port=35835
--webserver_interface=127.24.76.65
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:33281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:41:41.174927 17498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6: Bootstrap starting.
I20260808 11:41:41.194960 17498 log.cc:824] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6: Log is configured to *not* fsync() on all Append() calls
I20260808 11:41:41.207162 17498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 11:41:41.207685 17498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6: Bootstrap complete.
I20260808 11:41:41.216794 17498 raft_consensus.cc:348] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861318038ac14ba6acca16718dbd7fb6" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 33899 } }
I20260808 11:41:41.217607 17498 raft_consensus.cc:729] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 861318038ac14ba6acca16718dbd7fb6, State: Initialized, Role: FOLLOWER
I20260808 11:41:41.218250 17498 consensus_queue.cc:260] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861318038ac14ba6acca16718dbd7fb6" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 33899 } }
I20260808 11:41:41.218493 17498 raft_consensus.cc:388] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:41:41.218606 17498 raft_consensus.cc:482] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:41:41.218763 17498 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [term 1 FOLLOWER]: Advancing to term 2
I20260808 11:41:41.222501 17498 raft_consensus.cc:504] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861318038ac14ba6acca16718dbd7fb6" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 33899 } }
I20260808 11:41:41.222991 17498 leader_election.cc:302] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 861318038ac14ba6acca16718dbd7fb6; no voters:
I20260808 11:41:41.223616 17498 leader_election.cc:288] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 11:41:41.223763 17504 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 11:41:41.224928 17504 raft_consensus.cc:686] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [term 2 LEADER]: Becoming Leader. State: Replica: 861318038ac14ba6acca16718dbd7fb6, State: Running, Role: LEADER
I20260808 11:41:41.225499 17504 consensus_queue.cc:237] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861318038ac14ba6acca16718dbd7fb6" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 33899 } }
I20260808 11:41:41.226589 17498 sys_catalog.cc:564] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:41:41.227933 17505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "861318038ac14ba6acca16718dbd7fb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861318038ac14ba6acca16718dbd7fb6" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 33899 } } }
I20260808 11:41:41.228324 17505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [sys.catalog]: This master's current role is: LEADER
I20260808 11:41:41.230010 17515 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:41:41.229782 17510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 861318038ac14ba6acca16718dbd7fb6. Latest consensus state: current_term: 2 leader_uuid: "861318038ac14ba6acca16718dbd7fb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861318038ac14ba6acca16718dbd7fb6" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 33899 } } }
I20260808 11:41:41.230618 17510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [sys.catalog]: This master's current role is: LEADER
I20260808 11:41:41.238359 17515 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:41:41.239920 17515 catalog_manager.cc:1259] Loaded cluster ID: b3ccdb898f894d9faccd4d38e799eb5d
I20260808 11:41:41.240093 17515 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:41:41.244055 17515 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:41:41.245952 17515 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6: Loaded TSK: 0
I20260808 11:41:41.291222 17515 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6: Generated new TSK 1
I20260808 11:41:41.291596 17515 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 11:41:41.546875 17502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:41:41.547163 17502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:41:41.547298 17502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:41:41.554595 17502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:41:41.554706 17502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:41:41.554860 17502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.65
I20260808 11:41:41.564793 17502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:33281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.65:43595
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.76.65
--webserver_port=35835
--tserver_master_addrs=127.24.76.126:33899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.65
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:41:41.566156 17502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:41:41.567508 17502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:41:41.580762 17553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:41:41.581233 17556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:41:41.580809 17551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:41:41.581729 17502 server_base.cc:1034] running on GCE node
I20260808 11:41:41.582342 17502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:41:41.583264 17502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:41:41.584476 17502 hybrid_clock.cc:648] HybridClock initialized: now 1786189301584418 us; error 49 us; skew 500 ppm
I20260808 11:41:41.584774 17502 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:41:41 dist-test-slave-tc2s krb5kdc[16567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189301, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:41:41.588014 17502 init.cc:377] Logged in from keytab as kudu/127.24.76.65@KRBTEST.COM (short username kudu)
I20260808 11:41:41.589475 17502 webserver.cc:460] Webserver started at http://127.24.76.65:35835/ using document root <none> and password file <none>
I20260808 11:41:41.590209 17502 fs_manager.cc:359] Metadata directory not provided
I20260808 11:41:41.590354 17502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:41:41.595920 17502 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 11:41:41.598685 17567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:41:41.600135 17502 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:41:41.600307 17502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/wal
uuid: "f07b914ccc874645a5998194df57f1cc"
format_stamp: "Formatted at 2026-08-08 11:41:18 on dist-test-slave-tc2s"
server_key: "299ecc852022c57d5f1043b1eed43ca4"
server_key_iv: "020cb5d709c4d261c66596cf43dc0b9f"
server_key_version: "encryptionkey@0"
I20260808 11:41:41.601383 17502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:41:41.618252 17502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:41:41.622435 17502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:41:41.622694 17502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:41:41.623767 17502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:41:41.625108 17502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:41:41.625203 17502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:41:41.625308 17502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:41:41.625375 17502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:41:41.656610 17502 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.65:43595
I20260808 11:41:41.656674 17735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.65:43595 every 8 connection(s)
I20260808 11:41:41.658061 17502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
I20260808 11:41:41.663115 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 17502
I20260808 11:41:41.663935 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.66:33379
--local_ip_for_outbound_sockets=127.24.76.66
--tserver_master_addrs=127.24.76.126:33899
--webserver_port=33147
--webserver_interface=127.24.76.66
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:33281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 11:41:41 dist-test-slave-tc2s krb5kdc[16567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189301, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:41:41.683231 17741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:41.661167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.65:39215)
0808 11:41:41.662005 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:41.662060 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 11:41:41.664432 (+ 2372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:41:41.666095 (+ 1663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:41:41.666105 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:41:41.666793 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:41:41.667500 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:41:41.667523 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:41:41.669171 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:41:41.669176 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:41:41.670890 (+ 1714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:41:41.670906 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:41:41.671943 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:41:41.671950 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:41:41.672220 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:41.672893 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:41:41.672920 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:41:41.675466 (+ 2546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:41:41.679303 (+ 3837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:41:41.679310 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:41:41.679323 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:41:41.679745 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:41:41.680218 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:41:41.680222 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:41:41.680224 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:41:41.680430 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:41:41.681290 (+ 860us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:41:41.681296 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:41:41.681592 (+ 296us) client_negotiation.cc:769] Sending connection context
0808 11:41:41.681808 (+ 216us) client_negotiation.cc:240] Negotiation successful
0808 11:41:41.682238 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":338,"threads_started":1}
I20260808 11:41:41.683560 17739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:41.660933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:39215 (local address 127.24.76.126:33899)
0808 11:41:41.661536 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0808 11:41:41.661549 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 11:41:41.662184 (+ 635us) server_negotiation.cc:407] Connection header received
0808 11:41:41.665163 (+ 2979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:41:41.665177 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:41:41.665546 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:41:41.665851 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:41:41.667712 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:41.668997 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:41.671043 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:41.671702 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:41.671849 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:41.675740 (+ 3891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:41:41.675748 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:41:41.675762 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:41:41.675799 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:41:41.679151 (+ 3352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:41.679894 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:41.679902 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:41.679909 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:41.680111 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:41.680564 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:41.680569 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:41.680572 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:41.681154 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:41:41.681338 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0808 11:41:41.682097 (+ 759us) server_negotiation.cc:299] Negotiation successful
0808 11:41:41.682515 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":248,"threads_started":1}
I20260808 11:41:41.686350 17736 heartbeater.cc:347] Connected to a master server at 127.24.76.126:33899
I20260808 11:41:41.686697 17736 heartbeater.cc:464] Registering TS with master...
I20260808 11:41:41.687561 17736 heartbeater.cc:511] Master 127.24.76.126:33899 requested a full tablet report, sending...
I20260808 11:41:41.689930 17450 ts_manager.cc:194] Registered new tserver with Master: f07b914ccc874645a5998194df57f1cc (127.24.76.65:43595)
I20260808 11:41:41.691439 17450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.65@KRBTEST.COM'} at 127.24.76.65:39215
W20260808 11:41:42.006493 17742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:41:42.006768 17742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:41:42.006906 17742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:41:42.013813 17742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:41:42.013942 17742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:41:42.014106 17742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.66
I20260808 11:41:42.023394 17742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:33281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.66:33379
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.76.66
--webserver_port=33147
--tserver_master_addrs=127.24.76.126:33899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.66
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:41:42.024806 17742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:41:42.026222 17742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:41:42.038724 17751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:41:42.038775 17755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:41:42.038798 17750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:41:42.039538 17742 server_base.cc:1034] running on GCE node
I20260808 11:41:42.040131 17742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:41:42.041024 17742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:41:42.042217 17742 hybrid_clock.cc:648] HybridClock initialized: now 1786189302042170 us; error 36 us; skew 500 ppm
I20260808 11:41:42.042508 17742 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:41:42 dist-test-slave-tc2s krb5kdc[16567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189302, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:41:42.049783 17742 init.cc:377] Logged in from keytab as kudu/127.24.76.66@KRBTEST.COM (short username kudu)
I20260808 11:41:42.051144 17742 webserver.cc:460] Webserver started at http://127.24.76.66:33147/ using document root <none> and password file <none>
I20260808 11:41:42.051774 17742 fs_manager.cc:359] Metadata directory not provided
I20260808 11:41:42.051905 17742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:41:42.057107 17742 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 11:41:42.059670 17769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:41:42.060881 17742 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 11:41:42.061044 17742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/wal
uuid: "1d706a7ad7154186906a5efb55efd4ca"
format_stamp: "Formatted at 2026-08-08 11:41:19 on dist-test-slave-tc2s"
server_key: "883ccd7817cac0b8b831f0d12399bbe2"
server_key_iv: "3d759670ab5cbd662be1256c3286388c"
server_key_version: "encryptionkey@0"
I20260808 11:41:42.062007 17742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:41:42.084429 17742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:41:42.088266 17742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:41:42.088505 17742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:41:42.089409 17742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:41:42.090727 17742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:41:42.090816 17742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:41:42.090917 17742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:41:42.090991 17742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:41:42.118856 17742 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.66:33379
I20260808 11:41:42.118920 17938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.66:33379 every 8 connection(s)
I20260808 11:41:42.120198 17742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
I20260808 11:41:42.128347 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 17742
I20260808 11:41:42.128961 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.67:44575
--local_ip_for_outbound_sockets=127.24.76.67
--tserver_master_addrs=127.24.76.126:33899
--webserver_port=44925
--webserver_interface=127.24.76.67
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:33281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 11:41:42 dist-test-slave-tc2s krb5kdc[16567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189302, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:41:42.138825 17739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:42.122565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:47831 (local address 127.24.76.126:33899)
0808 11:41:42.122735 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 11:41:42.122740 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:41:42.123812 (+ 1072us) server_negotiation.cc:407] Connection header received
0808 11:41:42.125663 (+ 1851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:41:42.125668 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:41:42.125736 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:41:42.125814 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:41:42.127219 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.127988 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.129579 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.130037 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.130156 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:42.133548 (+ 3392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:41:42.133553 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:41:42.133556 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:41:42.133588 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:41:42.135865 (+ 2277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:42.136516 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:42.136522 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:42.136526 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:42.136643 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:42.137036 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:42.137041 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:42.137045 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:42.137304 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:41:42.137435 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 11:41:42.138275 (+ 840us) server_negotiation.cc:299] Negotiation successful
0808 11:41:42.138502 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 11:41:42.139585 17943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:42.122995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.66:47831)
0808 11:41:42.123669 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:42.123712 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 11:41:42.125400 (+ 1688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:41:42.126045 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:41:42.126053 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:41:42.126551 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:41:42.127089 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:41:42.127106 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.128144 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.128148 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:41:42.129432 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:41:42.129447 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.130260 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.130268 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:41:42.130476 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:42.131067 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:41:42.131091 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:41:42.133326 (+ 2235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:41:42.135997 (+ 2671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:41:42.136006 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:41:42.136021 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:41:42.136402 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:41:42.136744 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:41:42.136749 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:41:42.136753 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:41:42.136925 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:41:42.137427 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:41:42.137434 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:41:42.137726 (+ 292us) client_negotiation.cc:769] Sending connection context
0808 11:41:42.137999 (+ 273us) client_negotiation.cc:240] Negotiation successful
0808 11:41:42.138478 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":276,"threads_started":1}
I20260808 11:41:42.140882 17939 heartbeater.cc:347] Connected to a master server at 127.24.76.126:33899
I20260808 11:41:42.141170 17939 heartbeater.cc:464] Registering TS with master...
I20260808 11:41:42.141808 17939 heartbeater.cc:511] Master 127.24.76.126:33899 requested a full tablet report, sending...
I20260808 11:41:42.143225 17450 ts_manager.cc:194] Registered new tserver with Master: 1d706a7ad7154186906a5efb55efd4ca (127.24.76.66:33379)
I20260808 11:41:42.144229 17450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.66@KRBTEST.COM'} at 127.24.76.66:47831
W20260808 11:41:42.460628 17944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:41:42.460873 17944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:41:42.460963 17944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:41:42.467871 17944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:41:42.467965 17944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:41:42.468081 17944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.67
I20260808 11:41:42.477340 17944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:33281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.67:44575
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.76.67
--webserver_port=44925
--tserver_master_addrs=127.24.76.126:33899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.67
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:41:42.478497 17944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:41:42.479689 17944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:41:42.492249 17952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:41:42.492226 17954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:41:42.492319 17951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:41:42.492946 17944 server_base.cc:1034] running on GCE node
I20260808 11:41:42.493484 17944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:41:42.494340 17944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:41:42.495529 17944 hybrid_clock.cc:648] HybridClock initialized: now 1786189302495482 us; error 45 us; skew 500 ppm
I20260808 11:41:42.495805 17944 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:41:42 dist-test-slave-tc2s krb5kdc[16567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189302, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:41:42.498786 17944 init.cc:377] Logged in from keytab as kudu/127.24.76.67@KRBTEST.COM (short username kudu)
I20260808 11:41:42.500082 17944 webserver.cc:460] Webserver started at http://127.24.76.67:44925/ using document root <none> and password file <none>
I20260808 11:41:42.500732 17944 fs_manager.cc:359] Metadata directory not provided
I20260808 11:41:42.500865 17944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:41:42.506050 17944 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 11:41:42.508821 17965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:41:42.510080 17944 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:41:42.510236 17944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/wal
uuid: "f74fa58be8dd444eb55034cdc2ad57c9"
format_stamp: "Formatted at 2026-08-08 11:41:19 on dist-test-slave-tc2s"
server_key: "f178b819e5f67340e85b946763a65afa"
server_key_iv: "ab1acfed7b9269137915a17a89a21f37"
server_key_version: "encryptionkey@0"
I20260808 11:41:42.511164 17944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:41:42.525686 17944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:41:42.529572 17944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:41:42.529810 17944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:41:42.530732 17944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:41:42.531968 17944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:41:42.532074 17944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:41:42.532155 17944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:41:42.532213 17944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:41:42.561650 17944 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.67:44575
I20260808 11:41:42.561715 18171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.67:44575 every 8 connection(s)
I20260808 11:41:42.563056 17944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
I20260808 11:41:42.569137 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 17944
Aug 08 11:41:42 dist-test-slave-tc2s krb5kdc[16567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189302, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:41:42.582934 17739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:42.565562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:50817 (local address 127.24.76.126:33899)
0808 11:41:42.565757 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 11:41:42.565762 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:41:42.566926 (+ 1164us) server_negotiation.cc:407] Connection header received
0808 11:41:42.568703 (+ 1777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:41:42.568708 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:41:42.568793 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:41:42.568881 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:41:42.570712 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.571494 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.573001 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.573517 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.573627 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:42.577128 (+ 3501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:41:42.577135 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:41:42.577139 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:41:42.577169 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:41:42.580009 (+ 2840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:42.580711 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:42.580715 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:42.580717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:42.580838 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:42.581252 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:42.581255 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:42.581258 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:42.581525 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:41:42.581656 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 11:41:42.582354 (+ 698us) server_negotiation.cc:299] Negotiation successful
0808 11:41:42.582559 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":60}
I20260808 11:41:42.583353 18177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:42.570019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53472 (local address 127.24.76.126:33899)
0808 11:41:42.570475 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0808 11:41:42.570481 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:41:42.570775 (+ 294us) server_negotiation.cc:407] Connection header received
0808 11:41:42.570947 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:41:42.570951 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:41:42.571011 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:41:42.571108 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:41:42.572662 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.573391 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.574738 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.575144 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.575254 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:42.577960 (+ 2706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:41:42.577966 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:41:42.577971 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:41:42.578014 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:41:42.580690 (+ 2676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:42.581325 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:42.581330 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:42.581334 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:42.581463 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:42.581813 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:42.581818 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:42.581821 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:42.582099 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:41:42.582347 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0808 11:41:42.582632 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 11:41:42.582843 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":340,"thread_start_us":218,"threads_started":1}
I20260808 11:41:42.583918 18176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:42.566054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.67:50817)
0808 11:41:42.566750 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:42.566789 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 11:41:42.568447 (+ 1658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:41:42.569119 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:41:42.569127 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:41:42.569657 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:41:42.570488 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:41:42.570515 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.571653 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.571657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:41:42.572862 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:41:42.572875 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.573736 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.573744 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:41:42.574008 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:42.574615 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:41:42.574637 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:41:42.576936 (+ 2299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:41:42.580147 (+ 3211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:41:42.580156 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:41:42.580171 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:41:42.580588 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:41:42.580943 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:41:42.580948 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:41:42.580952 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:41:42.581136 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:41:42.581650 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:41:42.581657 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:41:42.582004 (+ 347us) client_negotiation.cc:769] Sending connection context
0808 11:41:42.582230 (+ 226us) client_negotiation.cc:240] Negotiation successful
0808 11:41:42.582723 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":265,"threads_started":1}
I20260808 11:41:42.585270 18174 heartbeater.cc:347] Connected to a master server at 127.24.76.126:33899
I20260808 11:41:42.585578 18174 heartbeater.cc:464] Registering TS with master...
I20260808 11:41:42.586316 18174 heartbeater.cc:511] Master 127.24.76.126:33899 requested a full tablet report, sending...
I20260808 11:41:42.587632 17449 ts_manager.cc:194] Registered new tserver with Master: f74fa58be8dd444eb55034cdc2ad57c9 (127.24.76.67:44575)
I20260808 11:41:42.588454 17449 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.67@KRBTEST.COM'} at 127.24.76.67:50817
I20260808 11:41:42.597851 24881 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:41:42.604241 18177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:42.598731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53478 (local address 127.24.76.126:33899)
0808 11:41:42.598880 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 11:41:42.598885 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:41:42.599385 (+ 500us) server_negotiation.cc:407] Connection header received
0808 11:41:42.599580 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:41:42.599584 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:41:42.599713 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:41:42.599829 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:41:42.600264 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.600944 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.602453 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.602854 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.602959 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:42.603452 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:41:42.603646 (+ 194us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 11:41:42.603968 (+ 322us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53478: token expired
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 11:41:42.604338 18177 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53478: token expired
W20260808 11:41:42.604910 17335 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.24.76.126:33899)
I20260808 11:41:42.616864 18177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:42.605381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53492 (local address 127.24.76.126:33899)
0808 11:41:42.605542 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 11:41:42.605547 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:41:42.605564 (+ 17us) server_negotiation.cc:407] Connection header received
0808 11:41:42.605733 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:41:42.605737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:41:42.605788 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:41:42.605860 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:41:42.607158 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.607826 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.609182 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.609599 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.609701 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:42.612120 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:41:42.612125 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:41:42.612128 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:41:42.612156 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:41:42.614369 (+ 2213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:42.614965 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:42.614970 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:42.614974 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:42.615123 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:42.615489 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:42.615495 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:42.615498 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:42.615787 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:41:42.616002 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0808 11:41:42.616344 (+ 342us) server_negotiation.cc:299] Negotiation successful
0808 11:41:42.616565 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 11:41:42.620131 17449 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53492:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:41:42.627383 17449 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:41:42.651031 18193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:42.638701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:33379 (local address 127.0.0.1:48150)
0808 11:41:42.640075 (+ 1374us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:42.640119 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 11:41:42.640319 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:41:42.641469 (+ 1150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:41:42.641477 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:41:42.641511 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:41:42.641943 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:41:42.641958 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.644237 (+ 2279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.644244 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:41:42.646984 (+ 2740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:41:42.647006 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.649675 (+ 2669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.649682 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:41:42.650203 (+ 521us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:42.650270 (+ 67us) client_negotiation.cc:769] Sending connection context
0808 11:41:42.650411 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 11:41:42.650609 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1113,"thread_start_us":960,"threads_started":1}
I20260808 11:41:42.651968 18195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:42.639024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48150 (local address 127.24.76.66:33379)
0808 11:41:42.639653 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0808 11:41:42.639663 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:41:42.640231 (+ 568us) server_negotiation.cc:407] Connection header received
0808 11:41:42.640612 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:41:42.640620 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:41:42.640868 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:41:42.641076 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:41:42.642189 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.644061 (+ 1872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.647204 (+ 3143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.648993 (+ 1789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.649124 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:42.649342 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0808 11:41:42.651442 (+ 2100us) server_negotiation.cc:299] Negotiation successful
0808 11:41:42.651629 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":313,"threads_started":1}
I20260808 11:41:42.652143 18194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:42.638945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:43595 (local address 127.0.0.1:34768)
0808 11:41:42.640693 (+ 1748us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:42.640707 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 11:41:42.640816 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:41:42.642405 (+ 1589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:41:42.642410 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:41:42.642438 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:41:42.642870 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:41:42.642881 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.646118 (+ 3237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.646124 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:41:42.648426 (+ 2302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:41:42.648443 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.651309 (+ 2866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.651316 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:41:42.651511 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:42.651547 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 11:41:42.651612 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 11:41:42.651769 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1653,"thread_start_us":1488,"threads_started":1}
I20260808 11:41:42.653672 18196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:42.639494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34768 (local address 127.24.76.65:43595)
0808 11:41:42.641795 (+ 2301us) server_negotiation.cc:206] Beginning negotiation
0808 11:41:42.641804 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:41:42.641833 (+ 29us) server_negotiation.cc:407] Connection header received
0808 11:41:42.641965 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:41:42.641972 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:41:42.642183 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:41:42.643086 (+ 903us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:41:42.643312 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.645955 (+ 2643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.649456 (+ 3501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.651059 (+ 1603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.652688 (+ 1629us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:42.653071 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0808 11:41:42.653151 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 11:41:42.653312 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2133,"thread_start_us":2015,"threads_started":1}
I20260808 11:41:42.656368 17844 tablet_service.cc:1467] Processing CreateTablet for tablet 3ce06e9b01db489c8266026af5920589 (DEFAULT_TABLE table=test-table [id=69a155df29d6423db027091c7cc2ca3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:41:42.658231 17844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ce06e9b01db489c8266026af5920589. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:41:42.661556 18194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:42.640825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:44575 (local address 127.0.0.1:60994)
0808 11:41:42.652382 (+ 11557us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:42.652400 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 11:41:42.652510 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:41:42.654867 (+ 2357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:41:42.654871 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:41:42.654896 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:41:42.655067 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:41:42.655077 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.657226 (+ 2149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.657231 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:41:42.659059 (+ 1828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:41:42.659075 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.660841 (+ 1766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.660847 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:41:42.661078 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:42.661109 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 11:41:42.661173 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 11:41:42.661308 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11434,"threads_started":1}
I20260808 11:41:42.662635 18200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:42.641144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60994 (local address 127.24.76.67:44575)
0808 11:41:42.654430 (+ 13286us) server_negotiation.cc:206] Beginning negotiation
0808 11:41:42.654436 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:41:42.654468 (+ 32us) server_negotiation.cc:407] Connection header received
0808 11:41:42.654543 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:41:42.654549 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:41:42.654695 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:41:42.654861 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:41:42.655307 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.657041 (+ 1734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.659261 (+ 2220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.660674 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.661831 (+ 1157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:42.662095 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0808 11:41:42.662164 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 11:41:42.662322 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13127,"thread_start_us":12980,"threads_started":1}
I20260808 11:41:42.666509 18047 tablet_service.cc:1467] Processing CreateTablet for tablet 3ce06e9b01db489c8266026af5920589 (DEFAULT_TABLE table=test-table [id=69a155df29d6423db027091c7cc2ca3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:41:42.666797 17639 tablet_service.cc:1467] Processing CreateTablet for tablet 3ce06e9b01db489c8266026af5920589 (DEFAULT_TABLE table=test-table [id=69a155df29d6423db027091c7cc2ca3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:41:42.667776 18047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ce06e9b01db489c8266026af5920589. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:41:42.668171 17639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ce06e9b01db489c8266026af5920589. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:41:42.674857 18203 tablet_bootstrap.cc:492] T 3ce06e9b01db489c8266026af5920589 P 1d706a7ad7154186906a5efb55efd4ca: Bootstrap starting.
I20260808 11:41:42.678507 18203 tablet_bootstrap.cc:654] T 3ce06e9b01db489c8266026af5920589 P 1d706a7ad7154186906a5efb55efd4ca: Neither blocks nor log segments found. Creating new log.
I20260808 11:41:42.679807 18203 log.cc:824] T 3ce06e9b01db489c8266026af5920589 P 1d706a7ad7154186906a5efb55efd4ca: Log is configured to *not* fsync() on all Append() calls
I20260808 11:41:42.680781 18204 tablet_bootstrap.cc:492] T 3ce06e9b01db489c8266026af5920589 P f74fa58be8dd444eb55034cdc2ad57c9: Bootstrap starting.
I20260808 11:41:42.681686 18206 tablet_bootstrap.cc:492] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc: Bootstrap starting.
I20260808 11:41:42.683920 18204 tablet_bootstrap.cc:654] T 3ce06e9b01db489c8266026af5920589 P f74fa58be8dd444eb55034cdc2ad57c9: Neither blocks nor log segments found. Creating new log.
I20260808 11:41:42.684723 18203 tablet_bootstrap.cc:492] T 3ce06e9b01db489c8266026af5920589 P 1d706a7ad7154186906a5efb55efd4ca: No bootstrap required, opened a new log
I20260808 11:41:42.684864 18204 log.cc:824] T 3ce06e9b01db489c8266026af5920589 P f74fa58be8dd444eb55034cdc2ad57c9: Log is configured to *not* fsync() on all Append() calls
I20260808 11:41:42.684999 18203 ts_tablet_manager.cc:1397] T 3ce06e9b01db489c8266026af5920589 P 1d706a7ad7154186906a5efb55efd4ca: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260808 11:41:42.685738 18206 tablet_bootstrap.cc:654] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc: Neither blocks nor log segments found. Creating new log.
I20260808 11:41:42.686933 18206 log.cc:824] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc: Log is configured to *not* fsync() on all Append() calls
I20260808 11:41:42.687812 18204 tablet_bootstrap.cc:492] T 3ce06e9b01db489c8266026af5920589 P f74fa58be8dd444eb55034cdc2ad57c9: No bootstrap required, opened a new log
I20260808 11:41:42.688367 18204 ts_tablet_manager.cc:1397] T 3ce06e9b01db489c8266026af5920589 P f74fa58be8dd444eb55034cdc2ad57c9: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260808 11:41:42.689885 18206 tablet_bootstrap.cc:492] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc: No bootstrap required, opened a new log
I20260808 11:41:42.690179 18206 ts_tablet_manager.cc:1397] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260808 11:41:42.692778 18203 raft_consensus.cc:348] T 3ce06e9b01db489c8266026af5920589 P 1d706a7ad7154186906a5efb55efd4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d706a7ad7154186906a5efb55efd4ca" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33379 } } peers { permanent_uuid: "f07b914ccc874645a5998194df57f1cc" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43595 } } peers { permanent_uuid: "f74fa58be8dd444eb55034cdc2ad57c9" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44575 } }
I20260808 11:41:42.693128 18203 raft_consensus.cc:374] T 3ce06e9b01db489c8266026af5920589 P 1d706a7ad7154186906a5efb55efd4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:41:42.693223 18203 raft_consensus.cc:729] T 3ce06e9b01db489c8266026af5920589 P 1d706a7ad7154186906a5efb55efd4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d706a7ad7154186906a5efb55efd4ca, State: Initialized, Role: FOLLOWER
I20260808 11:41:42.693745 18203 consensus_queue.cc:260] T 3ce06e9b01db489c8266026af5920589 P 1d706a7ad7154186906a5efb55efd4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d706a7ad7154186906a5efb55efd4ca" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33379 } } peers { permanent_uuid: "f07b914ccc874645a5998194df57f1cc" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43595 } } peers { permanent_uuid: "f74fa58be8dd444eb55034cdc2ad57c9" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44575 } }
I20260808 11:41:42.694784 17939 heartbeater.cc:503] Master 127.24.76.126:33899 was elected leader, sending a full tablet report...
I20260808 11:41:42.695024 17736 heartbeater.cc:503] Master 127.24.76.126:33899 was elected leader, sending a full tablet report...
I20260808 11:41:42.696559 18203 ts_tablet_manager.cc:1428] T 3ce06e9b01db489c8266026af5920589 P 1d706a7ad7154186906a5efb55efd4ca: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260808 11:41:42.699658 18204 raft_consensus.cc:348] T 3ce06e9b01db489c8266026af5920589 P f74fa58be8dd444eb55034cdc2ad57c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d706a7ad7154186906a5efb55efd4ca" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33379 } } peers { permanent_uuid: "f07b914ccc874645a5998194df57f1cc" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43595 } } peers { permanent_uuid: "f74fa58be8dd444eb55034cdc2ad57c9" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44575 } }
I20260808 11:41:42.700064 18204 raft_consensus.cc:374] T 3ce06e9b01db489c8266026af5920589 P f74fa58be8dd444eb55034cdc2ad57c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:41:42.700162 18204 raft_consensus.cc:729] T 3ce06e9b01db489c8266026af5920589 P f74fa58be8dd444eb55034cdc2ad57c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f74fa58be8dd444eb55034cdc2ad57c9, State: Initialized, Role: FOLLOWER
I20260808 11:41:42.700632 18206 raft_consensus.cc:348] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d706a7ad7154186906a5efb55efd4ca" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33379 } } peers { permanent_uuid: "f07b914ccc874645a5998194df57f1cc" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43595 } } peers { permanent_uuid: "f74fa58be8dd444eb55034cdc2ad57c9" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44575 } }
I20260808 11:41:42.700943 18206 raft_consensus.cc:374] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:41:42.700783 18204 consensus_queue.cc:260] T 3ce06e9b01db489c8266026af5920589 P f74fa58be8dd444eb55034cdc2ad57c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d706a7ad7154186906a5efb55efd4ca" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33379 } } peers { permanent_uuid: "f07b914ccc874645a5998194df57f1cc" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43595 } } peers { permanent_uuid: "f74fa58be8dd444eb55034cdc2ad57c9" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44575 } }
I20260808 11:41:42.701041 18206 raft_consensus.cc:729] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f07b914ccc874645a5998194df57f1cc, State: Initialized, Role: FOLLOWER
I20260808 11:41:42.701834 18174 heartbeater.cc:503] Master 127.24.76.126:33899 was elected leader, sending a full tablet report...
I20260808 11:41:42.702353 18204 ts_tablet_manager.cc:1428] T 3ce06e9b01db489c8266026af5920589 P f74fa58be8dd444eb55034cdc2ad57c9: Time spent starting tablet: real 0.014s user 0.008s sys 0.005s
I20260808 11:41:42.702564 18206 consensus_queue.cc:260] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d706a7ad7154186906a5efb55efd4ca" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33379 } } peers { permanent_uuid: "f07b914ccc874645a5998194df57f1cc" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43595 } } peers { permanent_uuid: "f74fa58be8dd444eb55034cdc2ad57c9" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44575 } }
I20260808 11:41:42.704036 18206 ts_tablet_manager.cc:1428] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc: Time spent starting tablet: real 0.014s user 0.007s sys 0.005s
I20260808 11:41:42.781194 18213 raft_consensus.cc:482] T 3ce06e9b01db489c8266026af5920589 P 1d706a7ad7154186906a5efb55efd4ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:41:42.781471 18213 raft_consensus.cc:504] T 3ce06e9b01db489c8266026af5920589 P 1d706a7ad7154186906a5efb55efd4ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d706a7ad7154186906a5efb55efd4ca" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33379 } } peers { permanent_uuid: "f07b914ccc874645a5998194df57f1cc" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43595 } } peers { permanent_uuid: "f74fa58be8dd444eb55034cdc2ad57c9" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44575 } }
I20260808 11:41:42.783116 18213 leader_election.cc:288] T 3ce06e9b01db489c8266026af5920589 P 1d706a7ad7154186906a5efb55efd4ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f07b914ccc874645a5998194df57f1cc (127.24.76.65:43595), f74fa58be8dd444eb55034cdc2ad57c9 (127.24.76.67:44575)
I20260808 11:41:42.789955 18196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:42.783587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:58575 (local address 127.24.76.65:43595)
0808 11:41:42.783829 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 11:41:42.783836 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:41:42.784187 (+ 351us) server_negotiation.cc:407] Connection header received
0808 11:41:42.784372 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:41:42.784378 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:41:42.784488 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:41:42.784581 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:41:42.785233 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.786188 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.788120 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.788979 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.789095 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:42.789146 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 11:41:42.789547 (+ 401us) server_negotiation.cc:299] Negotiation successful
0808 11:41:42.789671 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 11:41:42.790043 18219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:42.783462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:43595 (local address 127.24.76.66:58575)
0808 11:41:42.784074 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:42.784091 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:41:42.784234 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:41:42.784667 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:41:42.784672 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:41:42.784701 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:41:42.785073 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:41:42.785086 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.786314 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.786320 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:41:42.787966 (+ 1646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:41:42.787978 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.789190 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.789196 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:41:42.789371 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:42.789407 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 11:41:42.789494 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 11:41:42.789586 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"mutex_wait_us":47,"thread_start_us":336,"threads_started":1}
I20260808 11:41:42.790130 18200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:42.784476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:41243 (local address 127.24.76.67:44575)
0808 11:41:42.784654 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 11:41:42.784660 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:41:42.784872 (+ 212us) server_negotiation.cc:407] Connection header received
0808 11:41:42.785032 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:41:42.785038 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:41:42.785108 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:41:42.785183 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:41:42.785743 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.786558 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.788367 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.789201 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.789340 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:42.789407 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 11:41:42.789754 (+ 347us) server_negotiation.cc:299] Negotiation successful
0808 11:41:42.789877 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 11:41:42.790509 18220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:42.784183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:44575 (local address 127.24.76.66:41243)
0808 11:41:42.784778 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:42.784794 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:41:42.784913 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:41:42.785224 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:41:42.785228 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:41:42.785252 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:41:42.785607 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:41:42.785618 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.786679 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.786682 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:41:42.788217 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:41:42.788232 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.789401 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.789407 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:41:42.789577 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:42.789609 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 11:41:42.789701 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 11:41:42.789833 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"mutex_wait_us":35,"thread_start_us":259,"threads_started":1}
I20260808 11:41:42.791236 17664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ce06e9b01db489c8266026af5920589" candidate_uuid: "1d706a7ad7154186906a5efb55efd4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f07b914ccc874645a5998194df57f1cc" is_pre_election: true
I20260808 11:41:42.791450 18086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ce06e9b01db489c8266026af5920589" candidate_uuid: "1d706a7ad7154186906a5efb55efd4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f74fa58be8dd444eb55034cdc2ad57c9" is_pre_election: true
I20260808 11:41:42.791716 17664 raft_consensus.cc:2454] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d706a7ad7154186906a5efb55efd4ca in term 0.
I20260808 11:41:42.791925 18086 raft_consensus.cc:2454] T 3ce06e9b01db489c8266026af5920589 P f74fa58be8dd444eb55034cdc2ad57c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d706a7ad7154186906a5efb55efd4ca in term 0.
I20260808 11:41:42.792312 17771 leader_election.cc:302] T 3ce06e9b01db489c8266026af5920589 P 1d706a7ad7154186906a5efb55efd4ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d706a7ad7154186906a5efb55efd4ca, f07b914ccc874645a5998194df57f1cc; no voters:
I20260808 11:41:42.792651 18213 raft_consensus.cc:2781] T 3ce06e9b01db489c8266026af5920589 P 1d706a7ad7154186906a5efb55efd4ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:41:42.792809 18213 raft_consensus.cc:482] T 3ce06e9b01db489c8266026af5920589 P 1d706a7ad7154186906a5efb55efd4ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:41:42.792919 18213 raft_consensus.cc:3037] T 3ce06e9b01db489c8266026af5920589 P 1d706a7ad7154186906a5efb55efd4ca [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:41:42.795120 18213 raft_consensus.cc:504] T 3ce06e9b01db489c8266026af5920589 P 1d706a7ad7154186906a5efb55efd4ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d706a7ad7154186906a5efb55efd4ca" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33379 } } peers { permanent_uuid: "f07b914ccc874645a5998194df57f1cc" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43595 } } peers { permanent_uuid: "f74fa58be8dd444eb55034cdc2ad57c9" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44575 } }
I20260808 11:41:42.795663 18213 leader_election.cc:288] T 3ce06e9b01db489c8266026af5920589 P 1d706a7ad7154186906a5efb55efd4ca [CANDIDATE]: Term 1 election: Requested vote from peers f07b914ccc874645a5998194df57f1cc (127.24.76.65:43595), f74fa58be8dd444eb55034cdc2ad57c9 (127.24.76.67:44575)
I20260808 11:41:42.796058 17664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ce06e9b01db489c8266026af5920589" candidate_uuid: "1d706a7ad7154186906a5efb55efd4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f07b914ccc874645a5998194df57f1cc"
I20260808 11:41:42.796162 18086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ce06e9b01db489c8266026af5920589" candidate_uuid: "1d706a7ad7154186906a5efb55efd4ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f74fa58be8dd444eb55034cdc2ad57c9"
I20260808 11:41:42.796253 17664 raft_consensus.cc:3037] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:41:42.796351 18086 raft_consensus.cc:3037] T 3ce06e9b01db489c8266026af5920589 P f74fa58be8dd444eb55034cdc2ad57c9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:41:42.799431 17664 raft_consensus.cc:2454] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d706a7ad7154186906a5efb55efd4ca in term 1.
I20260808 11:41:42.799436 18086 raft_consensus.cc:2454] T 3ce06e9b01db489c8266026af5920589 P f74fa58be8dd444eb55034cdc2ad57c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d706a7ad7154186906a5efb55efd4ca in term 1.
I20260808 11:41:42.799845 17771 leader_election.cc:302] T 3ce06e9b01db489c8266026af5920589 P 1d706a7ad7154186906a5efb55efd4ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d706a7ad7154186906a5efb55efd4ca, f07b914ccc874645a5998194df57f1cc; no voters:
I20260808 11:41:42.800134 18213 raft_consensus.cc:2781] T 3ce06e9b01db489c8266026af5920589 P 1d706a7ad7154186906a5efb55efd4ca [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:41:42.800621 18213 raft_consensus.cc:686] T 3ce06e9b01db489c8266026af5920589 P 1d706a7ad7154186906a5efb55efd4ca [term 1 LEADER]: Becoming Leader. State: Replica: 1d706a7ad7154186906a5efb55efd4ca, State: Running, Role: LEADER
I20260808 11:41:42.801074 18213 consensus_queue.cc:237] T 3ce06e9b01db489c8266026af5920589 P 1d706a7ad7154186906a5efb55efd4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d706a7ad7154186906a5efb55efd4ca" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33379 } } peers { permanent_uuid: "f07b914ccc874645a5998194df57f1cc" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43595 } } peers { permanent_uuid: "f74fa58be8dd444eb55034cdc2ad57c9" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44575 } }
I20260808 11:41:42.805323 17449 catalog_manager.cc:5495] T 3ce06e9b01db489c8266026af5920589 P 1d706a7ad7154186906a5efb55efd4ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d706a7ad7154186906a5efb55efd4ca (127.24.76.66). New cstate: current_term: 1 leader_uuid: "1d706a7ad7154186906a5efb55efd4ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d706a7ad7154186906a5efb55efd4ca" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f07b914ccc874645a5998194df57f1cc" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f74fa58be8dd444eb55034cdc2ad57c9" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44575 } health_report { overall_health: UNKNOWN } } }
W20260808 11:41:42.814522 18175 tablet.cc:2369] T 3ce06e9b01db489c8266026af5920589 P f74fa58be8dd444eb55034cdc2ad57c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:41:42.853099 18195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:42.847902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48154 (local address 127.24.76.66:33379)
0808 11:41:42.848092 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 11:41:42.848097 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:41:42.848116 (+ 19us) server_negotiation.cc:407] Connection header received
0808 11:41:42.848291 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:41:42.848295 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:41:42.848376 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:41:42.848512 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:41:42.848997 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.849706 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.851153 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.851569 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.851657 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:42.852121 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:41:42.852603 (+ 482us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:41:42.852711 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 11:41:42.852792 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 11:41:42.852877 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 11:41:42.861502 17664 raft_consensus.cc:1261] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [term 1 FOLLOWER]: Refusing update from remote peer 1d706a7ad7154186906a5efb55efd4ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:41:42.861483 18086 raft_consensus.cc:1261] T 3ce06e9b01db489c8266026af5920589 P f74fa58be8dd444eb55034cdc2ad57c9 [term 1 FOLLOWER]: Refusing update from remote peer 1d706a7ad7154186906a5efb55efd4ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:41:42.862483 18221 consensus_queue.cc:1035] T 3ce06e9b01db489c8266026af5920589 P 1d706a7ad7154186906a5efb55efd4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "f74fa58be8dd444eb55034cdc2ad57c9" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:41:42.862831 18213 consensus_queue.cc:1035] T 3ce06e9b01db489c8266026af5920589 P 1d706a7ad7154186906a5efb55efd4ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "f07b914ccc874645a5998194df57f1cc" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 11:41:42.871937 17942 tablet.cc:2369] T 3ce06e9b01db489c8266026af5920589 P 1d706a7ad7154186906a5efb55efd4ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:41:42.872948 18228 mvcc.cc:205] Tried to move back new op lower bound from 7316231384510054400 to 7316231384276783104. Current Snapshot: MvccSnapshot[applied={T|T < 7316231384510054400}]
I20260808 11:41:42.873921 18234 mvcc.cc:205] Tried to move back new op lower bound from 7316231384510054400 to 7316231384276783104. Current Snapshot: MvccSnapshot[applied={T|T < 7316231384510054400}]
I20260808 11:41:42.886910 17449 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:41:42.887203 17449 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:41:42.890544 17449 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6: Sending DeleteTablet for 3 replicas of tablet 3ce06e9b01db489c8266026af5920589
I20260808 11:41:42.891628 17844 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ce06e9b01db489c8266026af5920589 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:41:42 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:48150
I20260808 11:41:42.891664 18047 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ce06e9b01db489c8266026af5920589 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:41:42 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:60994
I20260808 11:41:42.891709 17639 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ce06e9b01db489c8266026af5920589 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:41:42 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:34768
I20260808 11:41:42.892035 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 17502
I20260808 11:41:42.905162 18250 tablet_replica.cc:330] T 3ce06e9b01db489c8266026af5920589 P 1d706a7ad7154186906a5efb55efd4ca: stopping tablet replica
I20260808 11:41:42.905640 18250 raft_consensus.cc:2229] T 3ce06e9b01db489c8266026af5920589 P 1d706a7ad7154186906a5efb55efd4ca [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:41:42.906436 18250 raft_consensus.cc:2258] T 3ce06e9b01db489c8266026af5920589 P 1d706a7ad7154186906a5efb55efd4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:41:42.908849 18250 ts_tablet_manager.cc:1905] T 3ce06e9b01db489c8266026af5920589 P 1d706a7ad7154186906a5efb55efd4ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:41:42.911525 18196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:42.893512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34774 (local address 127.24.76.65:43595)
0808 11:41:42.893652 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 11:41:42.893657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:41:42.893674 (+ 17us) server_negotiation.cc:407] Connection header received
0808 11:41:42.893725 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:41:42.893730 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:41:42.893811 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:41:42.893945 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:41:42.895787 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.897001 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.898941 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:42.899607 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:42.899748 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:42.903135 (+ 3387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:41:42.903152 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:41:42.903169 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:41:42.903211 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:41:42.907312 (+ 4101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:42.908191 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:42.908201 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:42.908204 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:42.908363 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:42.908814 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:42.908820 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:42.908824 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:42.909346 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:41:42.909629 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0808 11:41:42.910888 (+ 1259us) server_negotiation.cc:299] Negotiation successful
0808 11:41:42.911133 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 11:41:42.911994 18253 tablet_replica.cc:330] T 3ce06e9b01db489c8266026af5920589 P f74fa58be8dd444eb55034cdc2ad57c9: stopping tablet replica
I20260808 11:41:42.912447 17730 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:41:42.912494 18253 raft_consensus.cc:2229] T 3ce06e9b01db489c8266026af5920589 P f74fa58be8dd444eb55034cdc2ad57c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:41:42.913012 18253 raft_consensus.cc:2258] T 3ce06e9b01db489c8266026af5920589 P f74fa58be8dd444eb55034cdc2ad57c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:41:42.911525 18251 tablet_replica.cc:330] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc: stopping tablet replica
I20260808 11:41:42.916364 18250 ts_tablet_manager.cc:1918] T 3ce06e9b01db489c8266026af5920589 P 1d706a7ad7154186906a5efb55efd4ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:41:42.916589 18250 log.cc:1196] T 3ce06e9b01db489c8266026af5920589 P 1d706a7ad7154186906a5efb55efd4ca: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/wal/wals/3ce06e9b01db489c8266026af5920589
I20260808 11:41:42.916582 18253 ts_tablet_manager.cc:1905] T 3ce06e9b01db489c8266026af5920589 P f74fa58be8dd444eb55034cdc2ad57c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:41:42.917300 18250 ts_tablet_manager.cc:1939] T 3ce06e9b01db489c8266026af5920589 P 1d706a7ad7154186906a5efb55efd4ca: Deleting consensus metadata
I20260808 11:41:42.921800 17426 catalog_manager.cc:4841] TS 1d706a7ad7154186906a5efb55efd4ca (127.24.76.66:33379): tablet 3ce06e9b01db489c8266026af5920589 (table test-table [id=69a155df29d6423db027091c7cc2ca3a]) successfully deleted
I20260808 11:41:42.927248 18253 ts_tablet_manager.cc:1918] T 3ce06e9b01db489c8266026af5920589 P f74fa58be8dd444eb55034cdc2ad57c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:41:42.927412 18253 log.cc:1196] T 3ce06e9b01db489c8266026af5920589 P f74fa58be8dd444eb55034cdc2ad57c9: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/wal/wals/3ce06e9b01db489c8266026af5920589
I20260808 11:41:42.927973 18253 ts_tablet_manager.cc:1939] T 3ce06e9b01db489c8266026af5920589 P f74fa58be8dd444eb55034cdc2ad57c9: Deleting consensus metadata
I20260808 11:41:42.929731 17423 catalog_manager.cc:4841] TS f74fa58be8dd444eb55034cdc2ad57c9 (127.24.76.67:44575): tablet 3ce06e9b01db489c8266026af5920589 (table test-table [id=69a155df29d6423db027091c7cc2ca3a]) successfully deleted
I20260808 11:41:42.913926 18251 raft_consensus.cc:2229] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:41:43.008214 18251 raft_consensus.cc:2258] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:41:43.008368 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 17502
W20260808 11:41:43.033319 17423 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.76.65:43595 (error 108)
I20260808 11:41:43.034025 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 17742
I20260808 11:41:43.035647 18197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:43.034928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:43595 (local address 127.0.0.1:34778)
0808 11:41:43.035212 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:43.035410 (+ 198us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:43595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154}
W20260808 11:41:43.035910 17423 catalog_manager.cc:4584] TS f07b914ccc874645a5998194df57f1cc (127.24.76.65:43595): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3ce06e9b01db489c8266026af5920589: Network error: Client connection negotiation failed: client connection to 127.24.76.65:43595: connect: Connection refused (error 111)
I20260808 11:41:43.049680 18195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:43.035045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48156 (local address 127.24.76.66:33379)
0808 11:41:43.035208 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 11:41:43.035214 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:41:43.035231 (+ 17us) server_negotiation.cc:407] Connection header received
0808 11:41:43.035296 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:41:43.035301 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:41:43.035384 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:41:43.035494 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:41:43.037474 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:43.038376 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:43.040245 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:43.040683 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:43.040773 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:43.044006 (+ 3233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:41:43.044020 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:41:43.044035 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:41:43.044067 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:41:43.046999 (+ 2932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:43.047651 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:43.047659 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:43.047665 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:43.047794 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:43.048150 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:43.048153 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:43.048155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:43.048529 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:41:43.048829 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0808 11:41:43.049126 (+ 297us) server_negotiation.cc:299] Negotiation successful
0808 11:41:43.049339 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 11:41:43.050391 17932 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:41:43.089341 18197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:43.088923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:43595 (local address 127.0.0.1:34786)
0808 11:41:43.089107 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:43.089197 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:43595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260808 11:41:43.090368 18197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:43.090058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:43595 (local address 127.0.0.1:34788)
0808 11:41:43.090177 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:43.090247 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:43595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 11:41:43.151094 18197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:43.150531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:43595 (local address 127.0.0.1:34804)
0808 11:41:43.150784 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:43.150915 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:43595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260808 11:41:43.152110 18197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:43.151766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:43595 (local address 127.0.0.1:34806)
0808 11:41:43.151907 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:43.151979 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:43595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 11:41:43.166745 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 17742
I20260808 11:41:43.196616 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 17944
I20260808 11:41:43.209141 18200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:43.197373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32778 (local address 127.24.76.67:44575)
0808 11:41:43.197580 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 11:41:43.197586 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:41:43.197605 (+ 19us) server_negotiation.cc:407] Connection header received
0808 11:41:43.197762 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:41:43.197765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:41:43.197832 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:41:43.197943 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:41:43.199329 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:43.200070 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:43.201237 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:41:43.201621 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:41:43.201709 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:41:43.204102 (+ 2393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:41:43.204114 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:41:43.204131 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:41:43.204167 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:41:43.206689 (+ 2522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:43.207400 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:43.207408 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:43.207410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:43.207555 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:41:43.207844 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:41:43.207847 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:41:43.207849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:41:43.208185 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:41:43.208397 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0808 11:41:43.208641 (+ 244us) server_negotiation.cc:299] Negotiation successful
0808 11:41:43.208803 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260808 11:41:43.209743 18158 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:41:43.232369 18197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:43.231837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:43595 (local address 127.0.0.1:34814)
0808 11:41:43.232072 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:43.232196 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:43595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 11:41:43.233598 18197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:43.233100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:43595 (local address 127.0.0.1:34826)
0808 11:41:43.233337 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:43.233423 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:43595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260808 11:41:43.298218 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 17944
I20260808 11:41:43.321847 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 17397
I20260808 11:41:43.323021 17491 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:41:43.437423 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 17397
I20260808 11:41:43.437497 18197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:41:43.436954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:43595 (local address 127.0.0.1:34836)
0808 11:41:43.437160 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 11:41:43.437271 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:43595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 11:42:03.467584 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:33899
--webserver_interface=127.24.76.126
--webserver_port=36373
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:33281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:33899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:42:03.806286 18265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:42:03.806571 18265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:42:03.806705 18265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:42:03.813673 18265 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 11:42:03.813786 18265 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 11:42:03.813853 18265 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 11:42:03.813951 18265 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:42:03.814023 18265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:42:03.814116 18265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:42:03.814172 18265 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:42:03.814246 18265 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:42:03.824038 18265 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:33281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:33899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:33899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=36373
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:42:03.825268 18265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:42:03.826609 18265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:42:03.838116 18273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:03.838074 18274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:03.838205 18282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:42:03.838492 18265 server_base.cc:1034] running on GCE node
I20260808 11:42:03.839475 18265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:42:03.840855 18265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:42:03.842100 18265 hybrid_clock.cc:648] HybridClock initialized: now 1786189323842041 us; error 29 us; skew 500 ppm
I20260808 11:42:03.842418 18265 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:42:03 dist-test-slave-tc2s krb5kdc[16567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189323, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:42:03.848217 18265 init.cc:377] Logged in from keytab as kudu/127.24.76.126@KRBTEST.COM (short username kudu)
I20260808 11:42:03.849674 18265 webserver.cc:460] Webserver started at http://127.24.76.126:36373/ using document root <none> and password file <none>
I20260808 11:42:03.850375 18265 fs_manager.cc:359] Metadata directory not provided
I20260808 11:42:03.850466 18265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:42:03.856077 18265 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260808 11:42:03.859017 18293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:03.860260 18265 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 11:42:03.860430 18265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "861318038ac14ba6acca16718dbd7fb6"
format_stamp: "Formatted at 2026-08-08 11:41:18 on dist-test-slave-tc2s"
server_key: "f5a9b17562f9d7e52d51fed9058ecc97"
server_key_iv: "d5caf735e621b5375e1ba3cf8385355d"
server_key_version: "encryptionkey@0"
I20260808 11:42:03.861344 18265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:42:03.894898 18265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:42:03.898993 18265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:42:03.899251 18265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:42:03.916666 18265 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:33899
I20260808 11:42:03.916702 18371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:33899 every 8 connection(s)
I20260808 11:42:03.918043 18265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:42:03.918752 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 18265
I20260808 11:42:03.919749 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.65:43595
--local_ip_for_outbound_sockets=127.24.76.65
--tserver_master_addrs=127.24.76.126:33899
--webserver_port=35835
--webserver_interface=127.24.76.65
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:33281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:42:03.925026 18376 sys_catalog.cc:263] Verifying existing consensus state
I20260808 11:42:03.928220 18376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6: Bootstrap starting.
I20260808 11:42:03.947803 18376 log.cc:824] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6: Log is configured to *not* fsync() on all Append() calls
I20260808 11:42:03.963522 18376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 11:42:03.964010 18376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6: Bootstrap complete.
I20260808 11:42:03.972877 18376 raft_consensus.cc:348] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861318038ac14ba6acca16718dbd7fb6" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 33899 } }
I20260808 11:42:03.973728 18376 raft_consensus.cc:729] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 861318038ac14ba6acca16718dbd7fb6, State: Initialized, Role: FOLLOWER
I20260808 11:42:03.974376 18376 consensus_queue.cc:260] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861318038ac14ba6acca16718dbd7fb6" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 33899 } }
I20260808 11:42:03.974591 18376 raft_consensus.cc:388] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:42:03.974699 18376 raft_consensus.cc:482] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:42:03.974859 18376 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [term 2 FOLLOWER]: Advancing to term 3
I20260808 11:42:03.978463 18376 raft_consensus.cc:504] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861318038ac14ba6acca16718dbd7fb6" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 33899 } }
I20260808 11:42:03.978906 18376 leader_election.cc:302] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 861318038ac14ba6acca16718dbd7fb6; no voters:
I20260808 11:42:03.979504 18376 leader_election.cc:288] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 11:42:03.979678 18381 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 11:42:03.980792 18381 raft_consensus.cc:686] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [term 3 LEADER]: Becoming Leader. State: Replica: 861318038ac14ba6acca16718dbd7fb6, State: Running, Role: LEADER
I20260808 11:42:03.981375 18381 consensus_queue.cc:237] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861318038ac14ba6acca16718dbd7fb6" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 33899 } }
I20260808 11:42:03.981556 18376 sys_catalog.cc:564] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:42:03.984450 18383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "861318038ac14ba6acca16718dbd7fb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861318038ac14ba6acca16718dbd7fb6" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 33899 } } }
I20260808 11:42:03.984771 18383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [sys.catalog]: This master's current role is: LEADER
I20260808 11:42:03.985437 18386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 861318038ac14ba6acca16718dbd7fb6. Latest consensus state: current_term: 3 leader_uuid: "861318038ac14ba6acca16718dbd7fb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "861318038ac14ba6acca16718dbd7fb6" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 33899 } } }
I20260808 11:42:03.985772 18386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6 [sys.catalog]: This master's current role is: LEADER
I20260808 11:42:03.986207 18389 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:42:03.996187 18389 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:42:03.997711 18389 catalog_manager.cc:1259] Loaded cluster ID: b3ccdb898f894d9faccd4d38e799eb5d
I20260808 11:42:03.997851 18389 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:42:04.002141 18389 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:42:04.003825 18389 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6: Loaded TSK: 0
I20260808 11:42:04.004009 18389 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 861318038ac14ba6acca16718dbd7fb6: Loaded TSK: 1
F20260808 11:42:04.005131 18389 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fd4a1e2c9ad google::LogMessage::Fail() at ??:0
@ 0x7fd4a1e2f88a google::LogMessage::SendToLog() at ??:0
@ 0x7fd4a1e2c4de google::LogMessage::Flush() at ??:0
@ 0x7fd4a1e2da19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fd4a5b907d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fd4a5b7f73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fd4b785b1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fd4b785a783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fd4b78e62a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7fd4b78e60c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fd4b79305e9 std::function<>::operator()() at ??:0
@ 0x7fd4b785ecdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fd4b785cc36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fd4b78e4610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fd4b78e4439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fd4a4a78345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fd4a4a7b91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fd4a4a7b799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fd4a4a56eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fd4a7d306db start_thread at ??:0
@ 0x7fd4a0cc661f clone at ??:0
W20260808 11:42:04.290491 18379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:42:04.290763 18379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:42:04.290895 18379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:42:04.298060 18379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:42:04.298177 18379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:42:04.298339 18379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.65
I20260808 11:42:04.308171 18379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:33281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.65:43595
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.76.65
--webserver_port=35835
--tserver_master_addrs=127.24.76.126:33899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.65
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:42:04.309423 18379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:42:04.310777 18379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:42:04.323573 18452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:04.323623 18456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:04.323539 18453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:42:04.324280 18379 server_base.cc:1034] running on GCE node
I20260808 11:42:04.324905 18379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:42:04.325758 18379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:42:04.327008 18379 hybrid_clock.cc:648] HybridClock initialized: now 1786189324326930 us; error 55 us; skew 500 ppm
I20260808 11:42:04.327318 18379 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:42:04 dist-test-slave-tc2s krb5kdc[16567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189324, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:42:04.330312 18379 init.cc:377] Logged in from keytab as kudu/127.24.76.65@KRBTEST.COM (short username kudu)
I20260808 11:42:04.331696 18379 webserver.cc:460] Webserver started at http://127.24.76.65:35835/ using document root <none> and password file <none>
I20260808 11:42:04.332351 18379 fs_manager.cc:359] Metadata directory not provided
I20260808 11:42:04.332469 18379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:42:04.337651 18379 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 11:42:04.340497 18464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:04.341701 18379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 11:42:04.341842 18379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/wal
uuid: "f07b914ccc874645a5998194df57f1cc"
format_stamp: "Formatted at 2026-08-08 11:41:18 on dist-test-slave-tc2s"
server_key: "299ecc852022c57d5f1043b1eed43ca4"
server_key_iv: "020cb5d709c4d261c66596cf43dc0b9f"
server_key_version: "encryptionkey@0"
I20260808 11:42:04.342803 18379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:42:04.366472 18379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:42:04.370236 18379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:42:04.370477 18379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:42:04.371424 18379 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:42:04.373231 18477 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260808 11:42:04.378760 18379 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 11:42:04.378875 18379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260808 11:42:04.378990 18379 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 11:42:04.382097 18379 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 11:42:04.382177 18379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.003s sys 0.000s
I20260808 11:42:04.382477 18477 tablet_bootstrap.cc:492] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc: Bootstrap starting.
I20260808 11:42:04.408126 18477 log.cc:824] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc: Log is configured to *not* fsync() on all Append() calls
I20260808 11:42:04.420910 18477 tablet_bootstrap.cc:492] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 11:42:04.421607 18477 tablet_bootstrap.cc:492] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc: Bootstrap complete.
I20260808 11:42:04.422053 18379 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.65:43595
I20260808 11:42:04.422081 18646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.65:43595 every 8 connection(s)
I20260808 11:42:04.422906 18477 ts_tablet_manager.cc:1397] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc: Time spent bootstrapping tablet: real 0.041s user 0.019s sys 0.019s
I20260808 11:42:04.423307 18379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
I20260808 11:42:04.428194 18649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:04.426038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.65:42593)
0808 11:42:04.426941 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:04.427289 (+ 348us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":391,"threads_started":1}
W20260808 11:42:04.428756 18647 heartbeater.cc:644] Failed to heartbeat to 127.24.76.126:33899 (0 consecutive failures): Network error: Failed to ping master at 127.24.76.126:33899: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
I20260808 11:42:04.429586 18649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:04.429267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.65:47205)
0808 11:42:04.429401 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:04.429473 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 11:42:04.430540 18649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:04.430117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.65:49759)
0808 11:42:04.430267 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:04.430363 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 11:42:04.430761 18647 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 11:42:04.430857 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 18379
I20260808 11:42:04.430487 18477 raft_consensus.cc:348] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d706a7ad7154186906a5efb55efd4ca" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33379 } } peers { permanent_uuid: "f07b914ccc874645a5998194df57f1cc" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43595 } } peers { permanent_uuid: "f74fa58be8dd444eb55034cdc2ad57c9" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44575 } }
I20260808 11:42:04.431526 18477 raft_consensus.cc:729] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f07b914ccc874645a5998194df57f1cc, State: Initialized, Role: FOLLOWER
I20260808 11:42:04.431620 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.66:33379
--local_ip_for_outbound_sockets=127.24.76.66
--tserver_master_addrs=127.24.76.126:33899
--webserver_port=33147
--webserver_interface=127.24.76.66
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:33281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:42:04.432091 18477 consensus_queue.cc:260] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d706a7ad7154186906a5efb55efd4ca" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33379 } } peers { permanent_uuid: "f07b914ccc874645a5998194df57f1cc" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43595 } } peers { permanent_uuid: "f74fa58be8dd444eb55034cdc2ad57c9" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44575 } }
I20260808 11:42:04.433439 18477 ts_tablet_manager.cc:1428] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc: Time spent starting tablet: real 0.010s user 0.001s sys 0.009s
I20260808 11:42:04.433638 18649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:04.433271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.65:42841)
0808 11:42:04.433432 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:04.433511 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 11:42:04.434458 18649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:04.434182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.65:33811)
0808 11:42:04.434293 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:04.434351 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 11:42:04.762820 18652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:42:04.763072 18652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:42:04.763170 18652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:42:04.770107 18652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:42:04.770197 18652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:42:04.770311 18652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.66
I20260808 11:42:04.779561 18652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:33281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.66:33379
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.76.66
--webserver_port=33147
--tserver_master_addrs=127.24.76.126:33899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.66
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:42:04.780710 18652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:42:04.781958 18652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:42:04.795116 18661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:04.795189 18660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:04.795158 18665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:42:04.795743 18652 server_base.cc:1034] running on GCE node
I20260808 11:42:04.796347 18652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:42:04.797226 18652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:42:04.798406 18652 hybrid_clock.cc:648] HybridClock initialized: now 1786189324798355 us; error 45 us; skew 500 ppm
I20260808 11:42:04.798705 18652 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:42:04 dist-test-slave-tc2s krb5kdc[16567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189324, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:42:04.801640 18652 init.cc:377] Logged in from keytab as kudu/127.24.76.66@KRBTEST.COM (short username kudu)
I20260808 11:42:04.803051 18652 webserver.cc:460] Webserver started at http://127.24.76.66:33147/ using document root <none> and password file <none>
I20260808 11:42:04.803712 18652 fs_manager.cc:359] Metadata directory not provided
I20260808 11:42:04.803831 18652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:42:04.808902 18652 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260808 11:42:04.811501 18679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:04.812711 18652 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 11:42:04.812863 18652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/wal
uuid: "1d706a7ad7154186906a5efb55efd4ca"
format_stamp: "Formatted at 2026-08-08 11:41:19 on dist-test-slave-tc2s"
server_key: "883ccd7817cac0b8b831f0d12399bbe2"
server_key_iv: "3d759670ab5cbd662be1256c3286388c"
server_key_version: "encryptionkey@0"
I20260808 11:42:04.813989 18652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:42:04.860971 18652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:42:04.865065 18652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:42:04.865322 18652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:42:04.866367 18652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:42:04.867726 18652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:42:04.867825 18652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:04.867936 18652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:42:04.868010 18652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:04.899032 18652 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.66:33379
I20260808 11:42:04.899115 18867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.66:33379 every 8 connection(s)
I20260808 11:42:04.900449 18652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
I20260808 11:42:04.901067 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 18652
I20260808 11:42:04.901808 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.67:44575
--local_ip_for_outbound_sockets=127.24.76.67
--tserver_master_addrs=127.24.76.126:33899
--webserver_port=44925
--webserver_interface=127.24.76.67
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:33281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:42:04.905669 18870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:04.903796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.66:39835)
0808 11:42:04.904566 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:04.904827 (+ 261us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":298,"threads_started":1}
W20260808 11:42:04.906327 18868 heartbeater.cc:644] Failed to heartbeat to 127.24.76.126:33899 (0 consecutive failures): Network error: Failed to ping master at 127.24.76.126:33899: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
I20260808 11:42:04.907091 18870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:04.906794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.66:38025)
0808 11:42:04.906917 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:04.906988 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 11:42:04.907866 18870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:04.907564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.66:33089)
0808 11:42:04.907693 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:04.907768 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 11:42:04.908088 18868 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 11:42:05.231251 18871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:42:05.231513 18871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:42:05.231650 18871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:42:05.238626 18871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:42:05.238722 18871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:42:05.238873 18871 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.67
I20260808 11:42:05.248458 18871 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:33281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.67:44575
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.76.67
--webserver_port=44925
--tserver_master_addrs=127.24.76.126:33899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.67
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:42:05.249780 18871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:42:05.251081 18871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:42:05.263830 18890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:05.263885 18888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:05.263911 18885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:42:05.264703 18871 server_base.cc:1034] running on GCE node
I20260808 11:42:05.265264 18871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:42:05.266173 18871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:42:05.267360 18871 hybrid_clock.cc:648] HybridClock initialized: now 1786189325267314 us; error 39 us; skew 500 ppm
I20260808 11:42:05.267640 18871 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:42:05 dist-test-slave-tc2s krb5kdc[16567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189325, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:42:05.270656 18871 init.cc:377] Logged in from keytab as kudu/127.24.76.67@KRBTEST.COM (short username kudu)
I20260808 11:42:05.271972 18871 webserver.cc:460] Webserver started at http://127.24.76.67:44925/ using document root <none> and password file <none>
I20260808 11:42:05.272588 18871 fs_manager.cc:359] Metadata directory not provided
I20260808 11:42:05.272688 18871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:42:05.277771 18871 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 11:42:05.280349 18899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:05.281571 18871 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 11:42:05.281718 18871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/wal
uuid: "f74fa58be8dd444eb55034cdc2ad57c9"
format_stamp: "Formatted at 2026-08-08 11:41:19 on dist-test-slave-tc2s"
server_key: "f178b819e5f67340e85b946763a65afa"
server_key_iv: "ab1acfed7b9269137915a17a89a21f37"
server_key_version: "encryptionkey@0"
I20260808 11:42:05.282718 18871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:42:05.306217 18871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:42:05.309981 18871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:42:05.310240 18871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:42:05.311285 18871 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:42:05.312618 18871 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:42:05.312716 18871 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:05.312824 18871 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:42:05.312889 18871 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:05.342971 18871 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.67:44575
I20260808 11:42:05.343044 19078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.67:44575 every 8 connection(s)
I20260808 11:42:05.344343 18871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
I20260808 11:42:05.349112 19087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:05.347336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.67:54973)
0808 11:42:05.348063 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:05.348313 (+ 250us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":276,"threads_started":1}
W20260808 11:42:05.349695 19083 heartbeater.cc:644] Failed to heartbeat to 127.24.76.126:33899 (0 consecutive failures): Network error: Failed to ping master at 127.24.76.126:33899: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
I20260808 11:42:05.350328 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 18871
I20260808 11:42:05.350546 19087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:05.350181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.67:42805)
0808 11:42:05.350335 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:05.350416 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 11:42:05.351361 19087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:05.351048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.67:54525)
0808 11:42:05.351181 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:05.351252 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 11:42:05.351625 19083 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 11:42:05.435910 19089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:05.435181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.65:47887)
0808 11:42:05.435628 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:05.435718 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":202,"threads_started":1}
I20260808 11:42:05.645694 19091 raft_consensus.cc:482] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:42:05.646018 19091 raft_consensus.cc:504] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d706a7ad7154186906a5efb55efd4ca" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33379 } } peers { permanent_uuid: "f07b914ccc874645a5998194df57f1cc" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43595 } } peers { permanent_uuid: "f74fa58be8dd444eb55034cdc2ad57c9" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44575 } }
I20260808 11:42:05.647684 19091 leader_election.cc:288] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1d706a7ad7154186906a5efb55efd4ca (127.24.76.66:33379), f74fa58be8dd444eb55034cdc2ad57c9 (127.24.76.67:44575)
Aug 08 11:42:05 dist-test-slave-tc2s krb5kdc[16567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189324, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for kudu/127.24.76.66@KRBTEST.COM
Aug 08 11:42:05 dist-test-slave-tc2s krb5kdc[16567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189324, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for kudu/127.24.76.67@KRBTEST.COM
I20260808 11:42:05.668030 19089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:05.647920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:33379 (local address 127.24.76.65:40247)
0808 11:42:05.648175 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:05.648220 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:05.648825 (+ 605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:05.652387 (+ 3562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:05.652392 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:05.653203 (+ 811us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:42:05.653980 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:05.654005 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:05.655590 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:05.655594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:05.657226 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:05.657238 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:05.658232 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:05.658240 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:05.658452 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:05.659238 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:42:05.659255 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:42:05.661447 (+ 2192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:42:05.664805 (+ 3358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:42:05.664815 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:42:05.664829 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:42:05.665337 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:42:05.665763 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:42:05.665768 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:42:05.665772 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:42:05.666047 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:42:05.666791 (+ 744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:42:05.666799 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:42:05.667093 (+ 294us) client_negotiation.cc:769] Sending connection context
0808 11:42:05.667397 (+ 304us) client_negotiation.cc:240] Negotiation successful
0808 11:42:05.667729 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":164,"spinlock_wait_cycles":51584}
I20260808 11:42:05.668298 19095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:05.648227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:40247 (local address 127.24.76.66:33379)
0808 11:42:05.650226 (+ 1999us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:05.650237 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:05.650269 (+ 32us) server_negotiation.cc:407] Connection header received
0808 11:42:05.651687 (+ 1418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:05.651696 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:05.652025 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:05.652432 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:05.654176 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:05.655399 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:05.657362 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:05.658040 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:05.658203 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:05.661727 (+ 3524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:05.661735 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:05.661748 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:42:05.661789 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:05.664662 (+ 2873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:05.665463 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:05.665469 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:05.665475 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:05.665644 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:05.666150 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:05.666155 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:05.666158 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:05.666632 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:05.666819 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:05.667668 (+ 849us) server_negotiation.cc:299] Negotiation successful
0808 11:42:05.667967 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1843,"thread_start_us":1721,"threads_started":1}
I20260808 11:42:05.669029 19093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:05.648013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:44575 (local address 127.24.76.65:40137)
0808 11:42:05.649611 (+ 1598us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:05.649629 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:05.649747 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:05.652328 (+ 2581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:05.652338 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:05.653189 (+ 851us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:42:05.653975 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:05.653997 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:05.655580 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:05.655584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:05.657218 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:05.657231 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:05.658212 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:05.658219 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:05.658448 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:05.659121 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:42:05.659159 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:42:05.661739 (+ 2580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:42:05.664845 (+ 3106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:42:05.664850 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:42:05.664855 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:42:05.665344 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:42:05.665760 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:42:05.665764 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:42:05.665766 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:42:05.666039 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:42:05.668237 (+ 2198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:42:05.668241 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:42:05.668372 (+ 131us) client_negotiation.cc:769] Sending connection context
0808 11:42:05.668503 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 11:42:05.668730 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1479,"mutex_wait_us":101,"thread_start_us":1265,"threads_started":1}
I20260808 11:42:05.669548 19096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:05.648401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:40137 (local address 127.24.76.67:44575)
0808 11:42:05.650018 (+ 1617us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:05.650029 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:05.650069 (+ 40us) server_negotiation.cc:407] Connection header received
0808 11:42:05.651534 (+ 1465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:05.651543 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:05.651892 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:05.652189 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:05.654157 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:05.655395 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:05.657346 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:05.658005 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:05.658159 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:05.662061 (+ 3902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:05.662068 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:05.662079 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:42:05.662114 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:05.664721 (+ 2607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:05.665478 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:05.665484 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:05.665489 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:05.665644 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:05.666142 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:05.666147 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:05.666149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:05.666649 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:05.666814 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:05.668763 (+ 1949us) server_negotiation.cc:299] Negotiation successful
0808 11:42:05.669156 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1402,"thread_start_us":1221,"threads_started":1}
W20260808 11:42:05.671284 18471 leader_election.cc:341] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1d706a7ad7154186906a5efb55efd4ca (127.24.76.66:33379): Not found: Tablet not found: 3ce06e9b01db489c8266026af5920589
W20260808 11:42:05.671857 18466 leader_election.cc:341] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f74fa58be8dd444eb55034cdc2ad57c9 (127.24.76.67:44575): Not found: Tablet not found: 3ce06e9b01db489c8266026af5920589
I20260808 11:42:05.671975 18466 leader_election.cc:302] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f07b914ccc874645a5998194df57f1cc; no voters: 1d706a7ad7154186906a5efb55efd4ca, f74fa58be8dd444eb55034cdc2ad57c9
I20260808 11:42:05.672312 19091 raft_consensus.cc:2726] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 11:42:05.909420 19097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:05.908575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.66:35329)
0808 11:42:05.909043 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:05.909199 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":213,"threads_started":1}
I20260808 11:42:06.352830 19098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:06.352102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.67:38673)
0808 11:42:06.352505 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:06.352632 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":191,"threads_started":1}
I20260808 11:42:06.437340 19100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:06.436701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.65:38209)
0808 11:42:06.437103 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:06.437181 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":204,"threads_started":1}
I20260808 11:42:06.910877 19101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:06.910220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.66:49109)
0808 11:42:06.910646 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:06.910725 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":225,"threads_started":1}
I20260808 11:42:07.354439 19102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:07.353644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.67:39189)
0808 11:42:07.354150 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:07.354253 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":227,"threads_started":1}
I20260808 11:42:07.438961 19103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:07.438172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.65:54089)
0808 11:42:07.438686 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:07.438768 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":229,"threads_started":1}
I20260808 11:42:07.771341 19105 raft_consensus.cc:482] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:42:07.771556 19105 raft_consensus.cc:504] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d706a7ad7154186906a5efb55efd4ca" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33379 } } peers { permanent_uuid: "f07b914ccc874645a5998194df57f1cc" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43595 } } peers { permanent_uuid: "f74fa58be8dd444eb55034cdc2ad57c9" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44575 } }
I20260808 11:42:07.772219 19105 leader_election.cc:288] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1d706a7ad7154186906a5efb55efd4ca (127.24.76.66:33379), f74fa58be8dd444eb55034cdc2ad57c9 (127.24.76.67:44575)
W20260808 11:42:07.773233 18471 leader_election.cc:341] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1d706a7ad7154186906a5efb55efd4ca (127.24.76.66:33379): Not found: Tablet not found: 3ce06e9b01db489c8266026af5920589
W20260808 11:42:07.773424 18466 leader_election.cc:341] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f74fa58be8dd444eb55034cdc2ad57c9 (127.24.76.67:44575): Not found: Tablet not found: 3ce06e9b01db489c8266026af5920589
I20260808 11:42:07.773576 18466 leader_election.cc:302] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f07b914ccc874645a5998194df57f1cc; no voters: 1d706a7ad7154186906a5efb55efd4ca, f74fa58be8dd444eb55034cdc2ad57c9
I20260808 11:42:07.773821 19105 raft_consensus.cc:2726] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 11:42:07.912397 19106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:07.911710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.66:50593)
0808 11:42:07.912158 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:07.912239 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":228,"threads_started":1}
I20260808 11:42:08.355911 19108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:08.355239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.67:54017)
0808 11:42:08.355655 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:08.355742 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":207,"threads_started":1}
I20260808 11:42:08.440755 19109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:08.439906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.65:52625)
0808 11:42:08.440478 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:08.440584 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":275,"threads_started":1}
I20260808 11:42:08.914038 19110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:08.913219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.66:56333)
0808 11:42:08.913715 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:08.913796 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":265,"threads_started":1}
I20260808 11:42:09.357589 19112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:09.356818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.67:40987)
0808 11:42:09.357274 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:09.357365 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":225,"threads_started":1}
I20260808 11:42:09.442523 19113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:09.441813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.65:45041)
0808 11:42:09.442249 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:09.442340 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":242,"threads_started":1}
I20260808 11:42:09.617113 19114 raft_consensus.cc:482] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:42:09.617318 19114 raft_consensus.cc:504] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d706a7ad7154186906a5efb55efd4ca" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33379 } } peers { permanent_uuid: "f07b914ccc874645a5998194df57f1cc" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43595 } } peers { permanent_uuid: "f74fa58be8dd444eb55034cdc2ad57c9" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44575 } }
I20260808 11:42:09.618037 19114 leader_election.cc:288] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1d706a7ad7154186906a5efb55efd4ca (127.24.76.66:33379), f74fa58be8dd444eb55034cdc2ad57c9 (127.24.76.67:44575)
W20260808 11:42:09.618899 18471 leader_election.cc:341] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1d706a7ad7154186906a5efb55efd4ca (127.24.76.66:33379): Not found: Tablet not found: 3ce06e9b01db489c8266026af5920589
W20260808 11:42:09.619089 18466 leader_election.cc:341] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f74fa58be8dd444eb55034cdc2ad57c9 (127.24.76.67:44575): Not found: Tablet not found: 3ce06e9b01db489c8266026af5920589
I20260808 11:42:09.619189 18466 leader_election.cc:302] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f07b914ccc874645a5998194df57f1cc; no voters: 1d706a7ad7154186906a5efb55efd4ca, f74fa58be8dd444eb55034cdc2ad57c9
I20260808 11:42:09.619392 19114 raft_consensus.cc:2726] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 11:42:09.915517 19115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:09.914832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.66:55727)
0808 11:42:09.915268 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:09.915348 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":244,"threads_started":1}
I20260808 11:42:10.359218 19117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:10.358514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.67:48939)
0808 11:42:10.358957 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:10.359062 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":243,"threads_started":1}
I20260808 11:42:10.444013 19120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:10.443331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.65:55509)
0808 11:42:10.443760 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:10.443839 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":240,"threads_started":1}
I20260808 11:42:10.916981 19121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:10.916258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.66:49169)
0808 11:42:10.916710 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:10.916789 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":228,"threads_started":1}
I20260808 11:42:11.360692 19122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:11.359973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.67:47107)
0808 11:42:11.360418 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:11.360509 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":220,"threads_started":1}
I20260808 11:42:11.445505 19124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:11.444781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.65:55445)
0808 11:42:11.445214 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:11.445307 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":229,"threads_started":1}
I20260808 11:42:11.918603 19125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:11.917774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.66:35575)
0808 11:42:11.918323 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:11.918403 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":288,"threads_started":1}
I20260808 11:42:12.362290 19126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:12.361507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.67:46503)
0808 11:42:12.362005 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:12.362085 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":213,"threads_started":1}
I20260808 11:42:12.447098 19127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:12.446335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.65:33379)
0808 11:42:12.446813 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:12.446892 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":273,"threads_started":1}
I20260808 11:42:12.920006 19128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:12.919350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.66:46677)
0808 11:42:12.919737 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:12.919815 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":194,"threads_started":1}
I20260808 11:42:13.363909 19130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:13.363124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.67:50147)
0808 11:42:13.363565 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:13.363656 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":221,"threads_started":1}
I20260808 11:42:13.448696 19132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:13.447961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.65:40073)
0808 11:42:13.448404 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:13.448511 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":228,"threads_started":1}
I20260808 11:42:13.800275 19133 raft_consensus.cc:482] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:42:13.800535 19133 raft_consensus.cc:504] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d706a7ad7154186906a5efb55efd4ca" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33379 } } peers { permanent_uuid: "f07b914ccc874645a5998194df57f1cc" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 43595 } } peers { permanent_uuid: "f74fa58be8dd444eb55034cdc2ad57c9" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44575 } }
I20260808 11:42:13.801168 19133 leader_election.cc:288] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1d706a7ad7154186906a5efb55efd4ca (127.24.76.66:33379), f74fa58be8dd444eb55034cdc2ad57c9 (127.24.76.67:44575)
W20260808 11:42:13.802189 18466 leader_election.cc:341] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f74fa58be8dd444eb55034cdc2ad57c9 (127.24.76.67:44575): Not found: Tablet not found: 3ce06e9b01db489c8266026af5920589
W20260808 11:42:13.802348 18471 leader_election.cc:341] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1d706a7ad7154186906a5efb55efd4ca (127.24.76.66:33379): Not found: Tablet not found: 3ce06e9b01db489c8266026af5920589
I20260808 11:42:13.802456 18471 leader_election.cc:302] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f07b914ccc874645a5998194df57f1cc; no voters: 1d706a7ad7154186906a5efb55efd4ca, f74fa58be8dd444eb55034cdc2ad57c9
I20260808 11:42:13.802712 19133 raft_consensus.cc:2726] T 3ce06e9b01db489c8266026af5920589 P f07b914ccc874645a5998194df57f1cc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 11:42:13.921510 19134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:13.920796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.66:38137)
0808 11:42:13.921268 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:13.921349 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":222,"threads_started":1}
I20260808 11:42:14.365506 19135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:14.364747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.67:43879)
0808 11:42:14.365251 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:14.365347 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":243,"threads_started":1}
I20260808 11:42:14.450340 19136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:14.449597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.65:40477)
0808 11:42:14.450085 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:14.450169 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":250,"threads_started":1}
I20260808 11:42:14.922996 19138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:14.922301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.66:55493)
0808 11:42:14.922728 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:14.922808 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":228,"threads_started":1}
I20260808 11:42:15.367075 19141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:15.366392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.67:42907)
0808 11:42:15.366806 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:15.366882 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":220,"threads_started":1}
I20260808 11:42:15.451992 19142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:15.451260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.65:46997)
0808 11:42:15.451712 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:15.451803 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":214,"threads_started":1}
I20260808 11:42:15.924502 19145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:15.923788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.66:34529)
0808 11:42:15.924231 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:15.924339 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":222,"threads_started":1}
I20260808 11:42:16.368603 19147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:16.367882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.67:40641)
0808 11:42:16.368350 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:16.368431 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":219,"threads_started":1}
I20260808 11:42:16.453693 19149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:16.453004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.65:34579)
0808 11:42:16.453438 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:16.453530 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":214,"threads_started":1}
I20260808 11:42:16.926086 19151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:16.925342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.66:41129)
0808 11:42:16.925779 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:16.925860 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":230,"threads_started":1}
I20260808 11:42:17.370217 19152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:17.369433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.67:50863)
0808 11:42:17.369951 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:17.370036 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":245,"threads_started":1}
I20260808 11:42:17.455504 19156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:17.454610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.65:38685)
0808 11:42:17.455157 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:17.455244 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":265,"threads_started":1}
I20260808 11:42:17.927603 19157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:17.926889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.66:54185)
0808 11:42:17.927338 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:17.927427 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":235,"threads_started":1}
I20260808 11:42:18.371799 19160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:18.371103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.67:43647)
0808 11:42:18.371559 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:18.371645 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":214,"threads_started":1}
I20260808 11:42:18.457142 19161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:18.456340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.65:50627)
0808 11:42:18.456841 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:18.456920 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":292,"threads_started":1}
I20260808 11:42:18.929188 19167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:18.928445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.66:60709)
0808 11:42:18.928875 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:18.928966 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":226,"threads_started":1}
I20260808 11:42:19.373301 19168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:19.372568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.67:38565)
0808 11:42:19.373031 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:19.373112 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":232,"threads_started":1}
I20260808 11:42:19.458941 19170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:19.458210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.65:35481)
0808 11:42:19.458659 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:19.458762 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":217,"threads_started":1}
I20260808 11:42:19.930927 19171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:19.930231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.66:38277)
0808 11:42:19.930659 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:19.930757 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":218,"threads_started":1}
I20260808 11:42:20.355835 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 18379
I20260808 11:42:20.369928 19172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:20.356889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47962 (local address 127.24.76.65:43595)
0808 11:42:20.357570 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:20.357580 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:20.357614 (+ 34us) server_negotiation.cc:407] Connection header received
0808 11:42:20.357703 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:20.357713 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:20.358040 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:20.358213 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:20.359562 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:20.360535 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:20.361684 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:20.362295 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:20.362389 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:20.364942 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:20.364952 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:20.364967 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:42:20.364996 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:20.367589 (+ 2593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:20.368194 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:20.368200 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:20.368202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:20.368336 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:20.368625 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:20.368628 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:20.368629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:20.368964 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:20.369179 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:20.369417 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 11:42:20.369625 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":371,"threads_started":1}
I20260808 11:42:20.371172 18639 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:42:20.375171 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:20.374354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33899 (local address 127.24.76.67:43317)
0808 11:42:20.374826 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:20.374942 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.126:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":237,"threads_started":1}
I20260808 11:42:20.462101 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 18379
I20260808 11:42:20.486882 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 18652
I20260808 11:42:20.499722 19178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:20.487647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45864 (local address 127.24.76.66:33379)
0808 11:42:20.488087 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:20.488091 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:20.488108 (+ 17us) server_negotiation.cc:407] Connection header received
0808 11:42:20.488165 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:20.488168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:20.488226 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:20.488315 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:20.489621 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:20.490462 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:20.491638 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:20.492058 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:20.492143 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:20.494846 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:20.494851 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:20.494855 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:42:20.494887 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:20.497261 (+ 2374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:20.497875 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:20.497878 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:20.497881 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:20.498152 (+ 271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:20.498464 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:20.498467 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:20.498469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:20.498727 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:20.498920 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:20.499170 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 11:42:20.499343 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":240,"threads_started":1}
I20260808 11:42:20.500412 18861 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:42:20.582274 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 18652
I20260808 11:42:20.606529 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 18871
I20260808 11:42:20.618965 19180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:20.607303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44418 (local address 127.24.76.67:44575)
0808 11:42:20.607871 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:20.607875 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:20.607891 (+ 16us) server_negotiation.cc:407] Connection header received
0808 11:42:20.607948 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:20.607952 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:20.608022 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:20.608113 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:20.609381 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:20.610187 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:20.611373 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:20.611793 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:20.611884 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:20.614362 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:20.614367 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:20.614370 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:42:20.614401 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:20.616810 (+ 2409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:20.617385 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:20.617388 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:20.617391 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:20.617542 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:20.617813 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:20.617816 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:20.617818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:20.618072 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:20.618277 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:20.618499 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 11:42:20.618671 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":295,"threads_started":1}
I20260808 11:42:20.619616 19068 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:42:20.701442 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 18871
2026-08-08T11:42:20Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64500 ms)
[----------] 30 tests from SecurityITest (298064 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:42:20 dist-test-slave-tc2s krb5kdc[19188](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:42:20 dist-test-slave-tc2s krb5kdc[19188](info): set up 2 sockets
Aug 08 11:42:20 dist-test-slave-tc2s krb5kdc[19188](info): commencing operation
krb5kdc: starting...
W20260808 11:42:21.950366 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.202s user 0.004s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:42:22 dist-test-slave-tc2s krb5kdc[19188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:42:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:42:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
I20260808 11:42:22.255596 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:33875
--webserver_interface=127.24.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:45699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:33875
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:42:22.591543 19214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:42:22.591831 19214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:42:22.591964 19214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:42:22.599109 19214 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:42:22.599222 19214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:42:22.599298 19214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:42:22.599365 19214 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:42:22.599422 19214 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:42:22.609123 19214 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:45699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:33875
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:33875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:42:22.610427 19214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:42:22.611758 19214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:42:22.623618 19224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:22.623559 19222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:22.623739 19221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:42:22.624243 19214 server_base.cc:1034] running on GCE node
I20260808 11:42:22.625041 19214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:42:22.626353 19214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:42:22.627554 19214 hybrid_clock.cc:648] HybridClock initialized: now 1786189342627518 us; error 27 us; skew 500 ppm
I20260808 11:42:22.627876 19214 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:42:22 dist-test-slave-tc2s krb5kdc[19188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189342, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:42:22.633677 19214 init.cc:377] Logged in from keytab as kudu/127.24.76.126@KRBTEST.COM (short username kudu)
I20260808 11:42:22.635164 19214 webserver.cc:460] Webserver started at http://127.24.76.126:36683/ using document root <none> and password file <none>
I20260808 11:42:22.635824 19214 fs_manager.cc:359] Metadata directory not provided
I20260808 11:42:22.635941 19214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:42:22.636224 19214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:42:22.639271 19214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "36bf2f52cac34fbbb950cf546b0deea0"
format_stamp: "Formatted at 2026-08-08 11:42:22 on dist-test-slave-tc2s"
server_key: "014fec8d9ea1bfd104157bcf0fb126a6"
server_key_iv: "91c95ce9957f0ba2b254ec5372d53593"
server_key_version: "encryptionkey@0"
I20260808 11:42:22.640014 19214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "36bf2f52cac34fbbb950cf546b0deea0"
format_stamp: "Formatted at 2026-08-08 11:42:22 on dist-test-slave-tc2s"
server_key: "014fec8d9ea1bfd104157bcf0fb126a6"
server_key_iv: "91c95ce9957f0ba2b254ec5372d53593"
server_key_version: "encryptionkey@0"
I20260808 11:42:22.645236 19214 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260808 11:42:22.648577 19237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:22.649840 19214 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 11:42:22.650058 19214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "36bf2f52cac34fbbb950cf546b0deea0"
format_stamp: "Formatted at 2026-08-08 11:42:22 on dist-test-slave-tc2s"
server_key: "014fec8d9ea1bfd104157bcf0fb126a6"
server_key_iv: "91c95ce9957f0ba2b254ec5372d53593"
server_key_version: "encryptionkey@0"
I20260808 11:42:22.650243 19214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:42:22.669082 19214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:42:22.672923 19214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:42:22.673187 19214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:42:22.690806 19214 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:33875
I20260808 11:42:22.690830 19331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:33875 every 8 connection(s)
I20260808 11:42:22.692116 19214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:42:22.694959 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 19214
I20260808 11:42:22.695190 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:42:22.696030 24881 external_mini_cluster.cc:1428] Setting key 2b65c6a7b48b95fb2e3f51e5259b0c8c
I20260808 11:42:22.696247 19332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:42:22.710129 19332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36bf2f52cac34fbbb950cf546b0deea0: Bootstrap starting.
Aug 08 11:42:22 dist-test-slave-tc2s krb5kdc[19188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:42:22.713259 19332 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36bf2f52cac34fbbb950cf546b0deea0: Neither blocks nor log segments found. Creating new log.
I20260808 11:42:22.714452 19332 log.cc:824] T 00000000000000000000000000000000 P 36bf2f52cac34fbbb950cf546b0deea0: Log is configured to *not* fsync() on all Append() calls
I20260808 11:42:22.717170 19332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36bf2f52cac34fbbb950cf546b0deea0: No bootstrap required, opened a new log
I20260808 11:42:22.719560 19335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:22.698168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47376 (local address 127.24.76.126:33875)
0808 11:42:22.698970 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:22.698984 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:22.699025 (+ 41us) server_negotiation.cc:407] Connection header received
0808 11:42:22.700905 (+ 1880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:22.700928 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:22.701311 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:22.701629 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:22.703449 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:22.704702 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:22.706514 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:22.707225 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:22.707374 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:22.712146 (+ 4772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:22.712153 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:22.712167 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:42:22.712202 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:22.715540 (+ 3338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:22.716264 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:22.716270 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:22.716275 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:22.716460 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:22.716834 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:22.716838 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:22.716840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:22.717397 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:22.717701 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:22.718158 (+ 457us) server_negotiation.cc:299] Negotiation successful
0808 11:42:22.718558 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":363,"threads_started":1}
I20260808 11:42:22.724982 19332 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36bf2f52cac34fbbb950cf546b0deea0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36bf2f52cac34fbbb950cf546b0deea0" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 33875 } }
I20260808 11:42:22.725306 19332 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36bf2f52cac34fbbb950cf546b0deea0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:42:22.725405 19332 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36bf2f52cac34fbbb950cf546b0deea0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36bf2f52cac34fbbb950cf546b0deea0, State: Initialized, Role: FOLLOWER
I20260808 11:42:22.726043 19332 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36bf2f52cac34fbbb950cf546b0deea0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36bf2f52cac34fbbb950cf546b0deea0" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 33875 } }
I20260808 11:42:22.726280 19332 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36bf2f52cac34fbbb950cf546b0deea0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:42:22.726459 19332 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36bf2f52cac34fbbb950cf546b0deea0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:42:22.726645 19332 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36bf2f52cac34fbbb950cf546b0deea0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:42:22.728657 19332 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36bf2f52cac34fbbb950cf546b0deea0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36bf2f52cac34fbbb950cf546b0deea0" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 33875 } }
I20260808 11:42:22.729076 19332 leader_election.cc:302] T 00000000000000000000000000000000 P 36bf2f52cac34fbbb950cf546b0deea0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36bf2f52cac34fbbb950cf546b0deea0; no voters:
I20260808 11:42:22.729588 19332 leader_election.cc:288] T 00000000000000000000000000000000 P 36bf2f52cac34fbbb950cf546b0deea0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:42:22.729714 19337 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36bf2f52cac34fbbb950cf546b0deea0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:42:22.730346 19337 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36bf2f52cac34fbbb950cf546b0deea0 [term 1 LEADER]: Becoming Leader. State: Replica: 36bf2f52cac34fbbb950cf546b0deea0, State: Running, Role: LEADER
I20260808 11:42:22.730866 19337 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36bf2f52cac34fbbb950cf546b0deea0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36bf2f52cac34fbbb950cf546b0deea0" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 33875 } }
I20260808 11:42:22.731580 19332 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36bf2f52cac34fbbb950cf546b0deea0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:42:22.733479 19339 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36bf2f52cac34fbbb950cf546b0deea0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36bf2f52cac34fbbb950cf546b0deea0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36bf2f52cac34fbbb950cf546b0deea0" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 33875 } } }
I20260808 11:42:22.734067 19339 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36bf2f52cac34fbbb950cf546b0deea0 [sys.catalog]: This master's current role is: LEADER
I20260808 11:42:22.733505 19341 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36bf2f52cac34fbbb950cf546b0deea0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36bf2f52cac34fbbb950cf546b0deea0. Latest consensus state: current_term: 1 leader_uuid: "36bf2f52cac34fbbb950cf546b0deea0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36bf2f52cac34fbbb950cf546b0deea0" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 33875 } } }
I20260808 11:42:22.734829 19341 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36bf2f52cac34fbbb950cf546b0deea0 [sys.catalog]: This master's current role is: LEADER
I20260808 11:42:22.735682 19349 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:42:22.740226 19349 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:42:22.753067 19349 catalog_manager.cc:1347] Generated new cluster ID: 85137ecd047d4fdbb66b5a63f0524d75
I20260808 11:42:22.753176 19349 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:42:22.773857 19349 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:42:22.775202 19349 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:42:22.786113 19349 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36bf2f52cac34fbbb950cf546b0deea0: Generated new TSK 0
I20260808 11:42:22.786865 19349 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab.
Entry for principal kudu/127.24.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab.
I20260808 11:42:22.871570 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.65:0
--local_ip_for_outbound_sockets=127.24.76.65
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:33875
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:45699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:42:23.211067 19371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:42:23.211359 19371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:42:23.211504 19371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:42:23.218751 19371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:42:23.218871 19371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:42:23.219026 19371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.65
I20260808 11:42:23.228447 19371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:45699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:33875
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.65
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:42:23.229673 19371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:42:23.231000 19371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:42:23.243569 19382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:23.243592 19383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:23.243644 19386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:42:23.244452 19371 server_base.cc:1034] running on GCE node
I20260808 11:42:23.245005 19371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:42:23.245837 19371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:42:23.247072 19371 hybrid_clock.cc:648] HybridClock initialized: now 1786189343247027 us; error 34 us; skew 500 ppm
I20260808 11:42:23.247367 19371 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:42:23 dist-test-slave-tc2s krb5kdc[19188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189343, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:42:23.250386 19371 init.cc:377] Logged in from keytab as kudu/127.24.76.65@KRBTEST.COM (short username kudu)
I20260808 11:42:23.251696 19371 webserver.cc:460] Webserver started at http://127.24.76.65:45537/ using document root <none> and password file <none>
I20260808 11:42:23.252333 19371 fs_manager.cc:359] Metadata directory not provided
I20260808 11:42:23.252458 19371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:42:23.252751 19371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:42:23.255703 19371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-0/data/instance:
uuid: "a11ee888e7e2432a8554efbbcc3b7915"
format_stamp: "Formatted at 2026-08-08 11:42:23 on dist-test-slave-tc2s"
server_key: "7f2242c83861ec1244340d3b2009268a"
server_key_iv: "62cd4b670fdac9e4e5435f1ef05c9316"
server_key_version: "encryptionkey@0"
I20260808 11:42:23.256433 19371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance:
uuid: "a11ee888e7e2432a8554efbbcc3b7915"
format_stamp: "Formatted at 2026-08-08 11:42:23 on dist-test-slave-tc2s"
server_key: "7f2242c83861ec1244340d3b2009268a"
server_key_iv: "62cd4b670fdac9e4e5435f1ef05c9316"
server_key_version: "encryptionkey@0"
I20260808 11:42:23.261564 19371 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 11:42:23.265086 19399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:23.266378 19371 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:42:23.266533 19371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-0/wal
uuid: "a11ee888e7e2432a8554efbbcc3b7915"
format_stamp: "Formatted at 2026-08-08 11:42:23 on dist-test-slave-tc2s"
server_key: "7f2242c83861ec1244340d3b2009268a"
server_key_iv: "62cd4b670fdac9e4e5435f1ef05c9316"
server_key_version: "encryptionkey@0"
I20260808 11:42:23.266713 19371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:42:23.290381 19371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:42:23.294677 19371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:42:23.294965 19371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:42:23.295921 19371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:42:23.297434 19371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:42:23.297552 19371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:23.297681 19371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:42:23.297737 19371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:23.328840 19371 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.65:45473
I20260808 11:42:23.328908 19605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.65:45473 every 8 connection(s)
I20260808 11:42:23.330169 19371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
I20260808 11:42:23.331804 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 19371
I20260808 11:42:23.332033 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance
I20260808 11:42:23.332827 24881 external_mini_cluster.cc:1428] Setting key 550868e2124bc6386e1e27110a230ca0
Aug 08 11:42:23 dist-test-slave-tc2s krb5kdc[19188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189343, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:42:23.350864 19611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:23.333123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:57123 (local address 127.24.76.126:33875)
0808 11:42:23.333636 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:23.333643 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:23.335050 (+ 1407us) server_negotiation.cc:407] Connection header received
0808 11:42:23.336662 (+ 1612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:23.336667 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:23.336749 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:23.336847 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:23.338416 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:23.339150 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:23.340721 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:23.341175 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:23.341293 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:23.344900 (+ 3607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:23.344905 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:23.344908 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:42:23.344940 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:23.347472 (+ 2532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:23.348335 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:23.348341 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:23.348344 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:23.348501 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:23.348976 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:23.348978 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:23.348981 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:23.349234 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:23.349343 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:23.350251 (+ 908us) server_negotiation.cc:299] Negotiation successful
0808 11:42:23.350491 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":240,"threads_started":1}
I20260808 11:42:23.351672 19613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:23.333744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33875 (local address 127.24.76.65:57123)
0808 11:42:23.334612 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:23.334662 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:23.336401 (+ 1739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:23.337098 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:23.337107 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:23.337636 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:42:23.338248 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:23.338268 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:23.339303 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:23.339307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:23.340582 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:23.340595 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:23.341404 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:23.341412 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:23.341666 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:23.342331 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:42:23.342353 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:42:23.344680 (+ 2327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:42:23.347631 (+ 2951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:42:23.347641 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:42:23.347656 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:42:23.348222 (+ 566us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:42:23.348618 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:42:23.348623 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:42:23.348626 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:42:23.348880 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:42:23.349358 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:42:23.349366 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:42:23.349815 (+ 449us) client_negotiation.cc:769] Sending connection context
0808 11:42:23.350099 (+ 284us) client_negotiation.cc:240] Negotiation successful
0808 11:42:23.350533 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":355,"threads_started":1}
I20260808 11:42:23.353284 19608 heartbeater.cc:347] Connected to a master server at 127.24.76.126:33875
I20260808 11:42:23.353652 19608 heartbeater.cc:464] Registering TS with master...
I20260808 11:42:23.354542 19608 heartbeater.cc:511] Master 127.24.76.126:33875 requested a full tablet report, sending...
I20260808 11:42:23.356797 19262 ts_manager.cc:194] Registered new tserver with Master: a11ee888e7e2432a8554efbbcc3b7915 (127.24.76.65:45473)
I20260808 11:42:23.358300 19262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.65@KRBTEST.COM'} at 127.24.76.65:57123
WARNING: no policy specified for kudu/127.24.76.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab.
Entry for principal kudu/127.24.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab.
I20260808 11:42:23.401088 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.66:0
--local_ip_for_outbound_sockets=127.24.76.66
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:33875
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:45699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:42:23.730103 19617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:42:23.730356 19617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:42:23.730492 19617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:42:23.737515 19617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:42:23.737620 19617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:42:23.737767 19617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.66
I20260808 11:42:23.747299 19617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:45699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:33875
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.66
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:42:23.748524 19617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:42:23.749758 19617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:42:23.762301 19624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:23.762517 19623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:23.762832 19627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:42:23.763090 19617 server_base.cc:1034] running on GCE node
I20260808 11:42:23.763665 19617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:42:23.764608 19617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:42:23.765803 19617 hybrid_clock.cc:648] HybridClock initialized: now 1786189343765755 us; error 43 us; skew 500 ppm
I20260808 11:42:23.766126 19617 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:42:23 dist-test-slave-tc2s krb5kdc[19188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189343, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:42:23.769032 19617 init.cc:377] Logged in from keytab as kudu/127.24.76.66@KRBTEST.COM (short username kudu)
I20260808 11:42:23.770512 19617 webserver.cc:460] Webserver started at http://127.24.76.66:33941/ using document root <none> and password file <none>
I20260808 11:42:23.771168 19617 fs_manager.cc:359] Metadata directory not provided
I20260808 11:42:23.771281 19617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:42:23.771570 19617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:42:23.774449 19617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-1/data/instance:
uuid: "688a5a7673f64325b25705862a2ee86c"
format_stamp: "Formatted at 2026-08-08 11:42:23 on dist-test-slave-tc2s"
server_key: "d811f13d11e84b73ea8ec006a989baf5"
server_key_iv: "004cf210d61d090d618195d6efab54a2"
server_key_version: "encryptionkey@0"
I20260808 11:42:23.775189 19617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance:
uuid: "688a5a7673f64325b25705862a2ee86c"
format_stamp: "Formatted at 2026-08-08 11:42:23 on dist-test-slave-tc2s"
server_key: "d811f13d11e84b73ea8ec006a989baf5"
server_key_iv: "004cf210d61d090d618195d6efab54a2"
server_key_version: "encryptionkey@0"
I20260808 11:42:23.780177 19617 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 11:42:23.783486 19636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:23.784669 19617 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 11:42:23.784818 19617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-1/wal
uuid: "688a5a7673f64325b25705862a2ee86c"
format_stamp: "Formatted at 2026-08-08 11:42:23 on dist-test-slave-tc2s"
server_key: "d811f13d11e84b73ea8ec006a989baf5"
server_key_iv: "004cf210d61d090d618195d6efab54a2"
server_key_version: "encryptionkey@0"
I20260808 11:42:23.785005 19617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:42:23.796540 19617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:42:23.800119 19617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:42:23.800380 19617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:42:23.801251 19617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:42:23.802511 19617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:42:23.802589 19617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:23.802706 19617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:42:23.802759 19617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:23.832459 19617 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.66:34301
I20260808 11:42:23.832507 19820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.66:34301 every 8 connection(s)
I20260808 11:42:23.833732 19617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
I20260808 11:42:23.840816 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 19617
I20260808 11:42:23.840993 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance
I20260808 11:42:23.841459 24881 external_mini_cluster.cc:1428] Setting key f23bdb173bc26159c0a4ea2c83a390df
Aug 08 11:42:23 dist-test-slave-tc2s krb5kdc[19188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189343, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:42:23.854542 19611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:23.836257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:54427 (local address 127.24.76.126:33875)
0808 11:42:23.836469 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:23.836476 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:23.837637 (+ 1161us) server_negotiation.cc:407] Connection header received
0808 11:42:23.839477 (+ 1840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:23.839481 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:23.839554 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:23.839640 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:23.841309 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:23.842210 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:23.844157 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:23.844581 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:23.844691 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:23.848218 (+ 3527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:23.848223 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:23.848227 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:42:23.848264 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:23.851310 (+ 3046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:23.852129 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:23.852135 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:23.852138 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:23.852297 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:23.852764 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:23.852768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:23.852771 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:23.853126 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:23.853265 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:23.854001 (+ 736us) server_negotiation.cc:299] Negotiation successful
0808 11:42:23.854236 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 11:42:23.855453 19823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:23.836770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33875 (local address 127.24.76.66:54427)
0808 11:42:23.837467 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:23.837514 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:23.839220 (+ 1706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:23.839821 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:23.839829 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:23.840435 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:42:23.841124 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:23.841147 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:23.842347 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:23.842353 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:23.844011 (+ 1658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:23.844028 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:23.844815 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:23.844823 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:23.845048 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:23.845702 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:42:23.845723 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:42:23.847971 (+ 2248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:42:23.851465 (+ 3494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:42:23.851474 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:42:23.851488 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:42:23.852005 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:42:23.852457 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:42:23.852461 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:42:23.852464 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:42:23.852658 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:42:23.853230 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:42:23.853235 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:42:23.853610 (+ 375us) client_negotiation.cc:769] Sending connection context
0808 11:42:23.853841 (+ 231us) client_negotiation.cc:240] Negotiation successful
0808 11:42:23.854274 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":263,"threads_started":1}
I20260808 11:42:23.856829 19821 heartbeater.cc:347] Connected to a master server at 127.24.76.126:33875
I20260808 11:42:23.857136 19821 heartbeater.cc:464] Registering TS with master...
I20260808 11:42:23.857815 19821 heartbeater.cc:511] Master 127.24.76.126:33875 requested a full tablet report, sending...
I20260808 11:42:23.859382 19262 ts_manager.cc:194] Registered new tserver with Master: 688a5a7673f64325b25705862a2ee86c (127.24.76.66:34301)
I20260808 11:42:23.860149 19262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.66@KRBTEST.COM'} at 127.24.76.66:54427
WARNING: no policy specified for kudu/127.24.76.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab.
Entry for principal kudu/127.24.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab.
I20260808 11:42:23.908767 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.67:0
--local_ip_for_outbound_sockets=127.24.76.67
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:33875
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:45699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:42:24.239692 19827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:42:24.239967 19827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:42:24.240090 19827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:42:24.247226 19827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:42:24.247325 19827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:42:24.247442 19827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.67
I20260808 11:42:24.256839 19827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:45699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:33875
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.67
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:42:24.258074 19827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:42:24.259290 19827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:42:24.271893 19834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:24.271888 19836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:24.272075 19842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:42:24.272904 19827 server_base.cc:1034] running on GCE node
I20260808 11:42:24.273509 19827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:42:24.274462 19827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:42:24.275655 19827 hybrid_clock.cc:648] HybridClock initialized: now 1786189344275604 us; error 39 us; skew 500 ppm
I20260808 11:42:24.275940 19827 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:42:24 dist-test-slave-tc2s krb5kdc[19188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189344, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:42:24.279026 19827 init.cc:377] Logged in from keytab as kudu/127.24.76.67@KRBTEST.COM (short username kudu)
I20260808 11:42:24.280452 19827 webserver.cc:460] Webserver started at http://127.24.76.67:41129/ using document root <none> and password file <none>
I20260808 11:42:24.281111 19827 fs_manager.cc:359] Metadata directory not provided
I20260808 11:42:24.281227 19827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:42:24.281528 19827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:42:24.284602 19827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-2/data/instance:
uuid: "43ebcfbe75894bfe9980098f8b1bba5f"
format_stamp: "Formatted at 2026-08-08 11:42:24 on dist-test-slave-tc2s"
server_key: "bd13937bae99263737229ef4128d81cc"
server_key_iv: "94c04eba98e4c08c0dd43d3528f5edc4"
server_key_version: "encryptionkey@0"
I20260808 11:42:24.285341 19827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance:
uuid: "43ebcfbe75894bfe9980098f8b1bba5f"
format_stamp: "Formatted at 2026-08-08 11:42:24 on dist-test-slave-tc2s"
server_key: "bd13937bae99263737229ef4128d81cc"
server_key_iv: "94c04eba98e4c08c0dd43d3528f5edc4"
server_key_version: "encryptionkey@0"
I20260808 11:42:24.290395 19827 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260808 11:42:24.293684 19852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:24.294925 19827 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:42:24.295075 19827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-2/wal
uuid: "43ebcfbe75894bfe9980098f8b1bba5f"
format_stamp: "Formatted at 2026-08-08 11:42:24 on dist-test-slave-tc2s"
server_key: "bd13937bae99263737229ef4128d81cc"
server_key_iv: "94c04eba98e4c08c0dd43d3528f5edc4"
server_key_version: "encryptionkey@0"
I20260808 11:42:24.295256 19827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:42:24.311831 19827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:42:24.315922 19827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:42:24.316190 19827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:42:24.317107 19827 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:42:24.318444 19827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:42:24.318521 19827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:24.318670 19827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:42:24.318724 19827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:24.348655 19827 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.67:40355
I20260808 11:42:24.348718 20034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.67:40355 every 8 connection(s)
I20260808 11:42:24.349831 19827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
I20260808 11:42:24.355469 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 19827
I20260808 11:42:24.355664 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance
I20260808 11:42:24.356247 24881 external_mini_cluster.cc:1428] Setting key 9739b95184b30c1d1d08b4de38a7abe6
I20260808 11:42:24.361720 19608 heartbeater.cc:503] Master 127.24.76.126:33875 was elected leader, sending a full tablet report...
Aug 08 11:42:24 dist-test-slave-tc2s krb5kdc[19188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189344, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:42:24.369670 19611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:24.352271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:59209 (local address 127.24.76.126:33875)
0808 11:42:24.352497 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:24.352503 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:24.353645 (+ 1142us) server_negotiation.cc:407] Connection header received
0808 11:42:24.355676 (+ 2031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:24.355682 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:24.355768 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:24.355860 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:24.357735 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:24.358713 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:24.360406 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:24.360795 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:24.360899 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:24.364317 (+ 3418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:24.364322 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:24.364325 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:42:24.364355 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:24.366895 (+ 2540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:24.367559 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:24.367564 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:24.367567 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:24.367689 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:24.368082 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:24.368087 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:24.368090 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:24.368338 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:24.368496 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:24.369128 (+ 632us) server_negotiation.cc:299] Negotiation successful
0808 11:42:24.369340 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 11:42:24.370747 20037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:24.352766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:33875 (local address 127.24.76.67:59209)
0808 11:42:24.353492 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:24.353530 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:24.355361 (+ 1831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:24.356139 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:24.356150 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:24.356876 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:42:24.357568 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:24.357594 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:24.358871 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:24.358877 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:24.360266 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:24.360279 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:24.360979 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:24.360984 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:24.361163 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:24.361768 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:42:24.361804 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:42:24.364128 (+ 2324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:42:24.367019 (+ 2891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:42:24.367028 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:42:24.367043 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:42:24.367447 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:42:24.367792 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:42:24.367797 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:42:24.367801 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:42:24.367978 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:42:24.368468 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:42:24.368476 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:42:24.368781 (+ 305us) client_negotiation.cc:769] Sending connection context
0808 11:42:24.369017 (+ 236us) client_negotiation.cc:240] Negotiation successful
0808 11:42:24.369523 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":269,"threads_started":1}
I20260808 11:42:24.372046 20035 heartbeater.cc:347] Connected to a master server at 127.24.76.126:33875
I20260808 11:42:24.372319 20035 heartbeater.cc:464] Registering TS with master...
I20260808 11:42:24.373000 20035 heartbeater.cc:511] Master 127.24.76.126:33875 requested a full tablet report, sending...
I20260808 11:42:24.374293 19262 ts_manager.cc:194] Registered new tserver with Master: 43ebcfbe75894bfe9980098f8b1bba5f (127.24.76.67:40355)
I20260808 11:42:24.375057 19262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.67@KRBTEST.COM'} at 127.24.76.67:59209
I20260808 11:42:24.384078 24881 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:42:24.401154 19611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:24.388900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47390 (local address 127.24.76.126:33875)
0808 11:42:24.389115 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:24.389122 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:24.389537 (+ 415us) server_negotiation.cc:407] Connection header received
0808 11:42:24.389680 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:24.389684 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:24.389747 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:24.389817 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:24.391325 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:24.392042 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:24.393430 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:24.393846 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:24.393983 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:24.396240 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:24.396245 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:24.396248 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:42:24.396278 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:24.398783 (+ 2505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:24.399407 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:24.399411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:24.399413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:24.399550 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:24.399881 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:24.399886 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:24.399889 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:24.400220 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:24.400415 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:24.400719 (+ 304us) server_negotiation.cc:299] Negotiation successful
0808 11:42:24.400878 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 11:42:24.405037 19262 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47390:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:42:24.411832 19262 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:42:24.434410 20060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:24.423955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48732 (local address 127.24.76.66:34301)
0808 11:42:24.426071 (+ 2116us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:24.426078 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:24.426105 (+ 27us) server_negotiation.cc:407] Connection header received
0808 11:42:24.426201 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:24.426209 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:24.426424 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:24.426791 (+ 367us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:42:24.427733 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:24.429304 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:24.431981 (+ 2677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:24.433400 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:24.433546 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:24.433724 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:24.433960 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 11:42:24.434124 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1948,"thread_start_us":1975,"threads_started":1}
I20260808 11:42:24.434517 20058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:24.423794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46262 (local address 127.24.76.65:45473)
0808 11:42:24.425763 (+ 1969us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:24.425771 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:24.425794 (+ 23us) server_negotiation.cc:407] Connection header received
0808 11:42:24.425878 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:24.425885 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:24.426163 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:24.427764 (+ 1601us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:42:24.427982 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:24.429616 (+ 1634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:24.431945 (+ 2329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:24.433008 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:24.433129 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:24.433368 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:24.434076 (+ 708us) server_negotiation.cc:299] Negotiation successful
0808 11:42:24.434222 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1814,"thread_start_us":1624,"threads_started":1}
I20260808 11:42:24.434725 20057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:24.423493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:45473 (local address 127.0.0.1:46262)
0808 11:42:24.424238 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:24.424280 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:24.424524 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:24.426423 (+ 1899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:24.426430 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:24.426462 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:42:24.426942 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:24.426956 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:24.429733 (+ 2777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:24.429738 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:24.431749 (+ 2011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:24.431762 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:24.433232 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:24.433239 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:24.433454 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:24.433502 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 11:42:24.433632 (+ 130us) client_negotiation.cc:240] Negotiation successful
0808 11:42:24.433800 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"mutex_wait_us":69,"thread_start_us":1317,"threads_started":1}
I20260808 11:42:24.435123 20056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:24.423613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:34301 (local address 127.0.0.1:48732)
0808 11:42:24.425071 (+ 1458us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:24.425092 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:24.425195 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:24.426641 (+ 1446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:24.426646 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:24.426673 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:42:24.427559 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:24.427569 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:24.429439 (+ 1870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:24.429444 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:24.431796 (+ 2352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:24.431812 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:24.433612 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:24.433619 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:24.433768 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:24.433797 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 11:42:24.433910 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 11:42:24.434063 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1336,"mutex_wait_us":362,"thread_start_us":492,"threads_started":1}
I20260808 11:42:24.437448 20061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:24.426238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:40355 (local address 127.0.0.1:46436)
0808 11:42:24.427572 (+ 1334us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:24.427590 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:24.427708 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:24.429807 (+ 2099us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:24.429811 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:24.429837 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:42:24.430416 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:24.430434 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:24.432342 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:24.432347 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:24.434875 (+ 2528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:24.434892 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:24.436679 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:24.436686 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:24.436856 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:24.436887 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 11:42:24.436975 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 11:42:24.437097 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1171,"mutex_wait_us":33,"thread_start_us":979,"threads_started":1}
I20260808 11:42:24.438167 20062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:24.426553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46436 (local address 127.24.76.67:40355)
0808 11:42:24.429296 (+ 2743us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:24.429302 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:24.429326 (+ 24us) server_negotiation.cc:407] Connection header received
0808 11:42:24.429403 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:24.429409 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:24.429620 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:24.429809 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:42:24.430620 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:24.432191 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:24.435037 (+ 2846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:24.436466 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:24.436596 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:24.436810 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:24.437750 (+ 940us) server_negotiation.cc:299] Negotiation successful
0808 11:42:24.437878 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2583,"thread_start_us":2441,"threads_started":1}
I20260808 11:42:24.439239 19721 tablet_service.cc:1467] Processing CreateTablet for tablet cf636e3b0c8f4b7984fa5123c38f7d87 (DEFAULT_TABLE table=test-table [id=dc238270df134f90b850b72e77797fb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:42:24.439584 19497 tablet_service.cc:1467] Processing CreateTablet for tablet cf636e3b0c8f4b7984fa5123c38f7d87 (DEFAULT_TABLE table=test-table [id=dc238270df134f90b850b72e77797fb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:42:24.440802 19721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf636e3b0c8f4b7984fa5123c38f7d87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:42:24.440903 19497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf636e3b0c8f4b7984fa5123c38f7d87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:42:24.440958 19931 tablet_service.cc:1467] Processing CreateTablet for tablet cf636e3b0c8f4b7984fa5123c38f7d87 (DEFAULT_TABLE table=test-table [id=dc238270df134f90b850b72e77797fb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:42:24.442174 19931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf636e3b0c8f4b7984fa5123c38f7d87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:42:24.453981 20063 tablet_bootstrap.cc:492] T cf636e3b0c8f4b7984fa5123c38f7d87 P 43ebcfbe75894bfe9980098f8b1bba5f: Bootstrap starting.
I20260808 11:42:24.455624 20066 tablet_bootstrap.cc:492] T cf636e3b0c8f4b7984fa5123c38f7d87 P a11ee888e7e2432a8554efbbcc3b7915: Bootstrap starting.
I20260808 11:42:24.457123 20069 tablet_bootstrap.cc:492] T cf636e3b0c8f4b7984fa5123c38f7d87 P 688a5a7673f64325b25705862a2ee86c: Bootstrap starting.
I20260808 11:42:24.458174 20063 tablet_bootstrap.cc:654] T cf636e3b0c8f4b7984fa5123c38f7d87 P 43ebcfbe75894bfe9980098f8b1bba5f: Neither blocks nor log segments found. Creating new log.
I20260808 11:42:24.458938 20066 tablet_bootstrap.cc:654] T cf636e3b0c8f4b7984fa5123c38f7d87 P a11ee888e7e2432a8554efbbcc3b7915: Neither blocks nor log segments found. Creating new log.
I20260808 11:42:24.459462 20063 log.cc:824] T cf636e3b0c8f4b7984fa5123c38f7d87 P 43ebcfbe75894bfe9980098f8b1bba5f: Log is configured to *not* fsync() on all Append() calls
I20260808 11:42:24.460026 20069 tablet_bootstrap.cc:654] T cf636e3b0c8f4b7984fa5123c38f7d87 P 688a5a7673f64325b25705862a2ee86c: Neither blocks nor log segments found. Creating new log.
I20260808 11:42:24.460500 20066 log.cc:824] T cf636e3b0c8f4b7984fa5123c38f7d87 P a11ee888e7e2432a8554efbbcc3b7915: Log is configured to *not* fsync() on all Append() calls
I20260808 11:42:24.460983 20069 log.cc:824] T cf636e3b0c8f4b7984fa5123c38f7d87 P 688a5a7673f64325b25705862a2ee86c: Log is configured to *not* fsync() on all Append() calls
I20260808 11:42:24.463577 20069 tablet_bootstrap.cc:492] T cf636e3b0c8f4b7984fa5123c38f7d87 P 688a5a7673f64325b25705862a2ee86c: No bootstrap required, opened a new log
I20260808 11:42:24.463668 20066 tablet_bootstrap.cc:492] T cf636e3b0c8f4b7984fa5123c38f7d87 P a11ee888e7e2432a8554efbbcc3b7915: No bootstrap required, opened a new log
I20260808 11:42:24.463776 20063 tablet_bootstrap.cc:492] T cf636e3b0c8f4b7984fa5123c38f7d87 P 43ebcfbe75894bfe9980098f8b1bba5f: No bootstrap required, opened a new log
I20260808 11:42:24.463832 20069 ts_tablet_manager.cc:1397] T cf636e3b0c8f4b7984fa5123c38f7d87 P 688a5a7673f64325b25705862a2ee86c: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 11:42:24.463959 20066 ts_tablet_manager.cc:1397] T cf636e3b0c8f4b7984fa5123c38f7d87 P a11ee888e7e2432a8554efbbcc3b7915: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260808 11:42:24.464005 20063 ts_tablet_manager.cc:1397] T cf636e3b0c8f4b7984fa5123c38f7d87 P 43ebcfbe75894bfe9980098f8b1bba5f: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260808 11:42:24.471215 20069 raft_consensus.cc:348] T cf636e3b0c8f4b7984fa5123c38f7d87 P 688a5a7673f64325b25705862a2ee86c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a11ee888e7e2432a8554efbbcc3b7915" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 45473 } } peers { permanent_uuid: "688a5a7673f64325b25705862a2ee86c" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 34301 } } peers { permanent_uuid: "43ebcfbe75894bfe9980098f8b1bba5f" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 40355 } }
I20260808 11:42:24.471602 20069 raft_consensus.cc:374] T cf636e3b0c8f4b7984fa5123c38f7d87 P 688a5a7673f64325b25705862a2ee86c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:42:24.471695 20069 raft_consensus.cc:729] T cf636e3b0c8f4b7984fa5123c38f7d87 P 688a5a7673f64325b25705862a2ee86c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 688a5a7673f64325b25705862a2ee86c, State: Initialized, Role: FOLLOWER
I20260808 11:42:24.472232 20069 consensus_queue.cc:260] T cf636e3b0c8f4b7984fa5123c38f7d87 P 688a5a7673f64325b25705862a2ee86c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a11ee888e7e2432a8554efbbcc3b7915" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 45473 } } peers { permanent_uuid: "688a5a7673f64325b25705862a2ee86c" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 34301 } } peers { permanent_uuid: "43ebcfbe75894bfe9980098f8b1bba5f" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 40355 } }
I20260808 11:42:24.473242 19821 heartbeater.cc:503] Master 127.24.76.126:33875 was elected leader, sending a full tablet report...
I20260808 11:42:24.473865 20069 ts_tablet_manager.cc:1428] T cf636e3b0c8f4b7984fa5123c38f7d87 P 688a5a7673f64325b25705862a2ee86c: Time spent starting tablet: real 0.010s user 0.005s sys 0.005s
I20260808 11:42:24.474076 20066 raft_consensus.cc:348] T cf636e3b0c8f4b7984fa5123c38f7d87 P a11ee888e7e2432a8554efbbcc3b7915 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a11ee888e7e2432a8554efbbcc3b7915" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 45473 } } peers { permanent_uuid: "688a5a7673f64325b25705862a2ee86c" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 34301 } } peers { permanent_uuid: "43ebcfbe75894bfe9980098f8b1bba5f" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 40355 } }
I20260808 11:42:24.474466 20066 raft_consensus.cc:374] T cf636e3b0c8f4b7984fa5123c38f7d87 P a11ee888e7e2432a8554efbbcc3b7915 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:42:24.474575 20066 raft_consensus.cc:729] T cf636e3b0c8f4b7984fa5123c38f7d87 P a11ee888e7e2432a8554efbbcc3b7915 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a11ee888e7e2432a8554efbbcc3b7915, State: Initialized, Role: FOLLOWER
I20260808 11:42:24.474866 20063 raft_consensus.cc:348] T cf636e3b0c8f4b7984fa5123c38f7d87 P 43ebcfbe75894bfe9980098f8b1bba5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a11ee888e7e2432a8554efbbcc3b7915" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 45473 } } peers { permanent_uuid: "688a5a7673f64325b25705862a2ee86c" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 34301 } } peers { permanent_uuid: "43ebcfbe75894bfe9980098f8b1bba5f" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 40355 } }
I20260808 11:42:24.475274 20063 raft_consensus.cc:374] T cf636e3b0c8f4b7984fa5123c38f7d87 P 43ebcfbe75894bfe9980098f8b1bba5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:42:24.475152 20066 consensus_queue.cc:260] T cf636e3b0c8f4b7984fa5123c38f7d87 P a11ee888e7e2432a8554efbbcc3b7915 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a11ee888e7e2432a8554efbbcc3b7915" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 45473 } } peers { permanent_uuid: "688a5a7673f64325b25705862a2ee86c" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 34301 } } peers { permanent_uuid: "43ebcfbe75894bfe9980098f8b1bba5f" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 40355 } }
I20260808 11:42:24.475390 20063 raft_consensus.cc:729] T cf636e3b0c8f4b7984fa5123c38f7d87 P 43ebcfbe75894bfe9980098f8b1bba5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43ebcfbe75894bfe9980098f8b1bba5f, State: Initialized, Role: FOLLOWER
I20260808 11:42:24.475960 20063 consensus_queue.cc:260] T cf636e3b0c8f4b7984fa5123c38f7d87 P 43ebcfbe75894bfe9980098f8b1bba5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a11ee888e7e2432a8554efbbcc3b7915" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 45473 } } peers { permanent_uuid: "688a5a7673f64325b25705862a2ee86c" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 34301 } } peers { permanent_uuid: "43ebcfbe75894bfe9980098f8b1bba5f" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 40355 } }
I20260808 11:42:24.476838 20066 ts_tablet_manager.cc:1428] T cf636e3b0c8f4b7984fa5123c38f7d87 P a11ee888e7e2432a8554efbbcc3b7915: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260808 11:42:24.478489 20035 heartbeater.cc:503] Master 127.24.76.126:33875 was elected leader, sending a full tablet report...
I20260808 11:42:24.478734 20063 ts_tablet_manager.cc:1428] T cf636e3b0c8f4b7984fa5123c38f7d87 P 43ebcfbe75894bfe9980098f8b1bba5f: Time spent starting tablet: real 0.015s user 0.011s sys 0.005s
W20260808 11:42:24.582430 19609 tablet.cc:2369] T cf636e3b0c8f4b7984fa5123c38f7d87 P a11ee888e7e2432a8554efbbcc3b7915: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:42:24.585585 19822 tablet.cc:2369] T cf636e3b0c8f4b7984fa5123c38f7d87 P 688a5a7673f64325b25705862a2ee86c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:42:24.601377 20036 tablet.cc:2369] T cf636e3b0c8f4b7984fa5123c38f7d87 P 43ebcfbe75894bfe9980098f8b1bba5f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:42:24.653779 20080 raft_consensus.cc:482] T cf636e3b0c8f4b7984fa5123c38f7d87 P a11ee888e7e2432a8554efbbcc3b7915 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:42:24.654125 20080 raft_consensus.cc:504] T cf636e3b0c8f4b7984fa5123c38f7d87 P a11ee888e7e2432a8554efbbcc3b7915 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a11ee888e7e2432a8554efbbcc3b7915" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 45473 } } peers { permanent_uuid: "688a5a7673f64325b25705862a2ee86c" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 34301 } } peers { permanent_uuid: "43ebcfbe75894bfe9980098f8b1bba5f" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 40355 } }
I20260808 11:42:24.655977 20080 leader_election.cc:288] T cf636e3b0c8f4b7984fa5123c38f7d87 P a11ee888e7e2432a8554efbbcc3b7915 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 688a5a7673f64325b25705862a2ee86c (127.24.76.66:34301), 43ebcfbe75894bfe9980098f8b1bba5f (127.24.76.67:40355)
I20260808 11:42:24.662851 20062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:24.656491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:54193 (local address 127.24.76.67:40355)
0808 11:42:24.656722 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:24.656729 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:24.657401 (+ 672us) server_negotiation.cc:407] Connection header received
0808 11:42:24.657572 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:24.657579 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:24.657661 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:24.657732 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:42:24.658451 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:24.659212 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:24.661143 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:24.661927 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:24.662076 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:24.662153 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:24.662495 (+ 342us) server_negotiation.cc:299] Negotiation successful
0808 11:42:24.662617 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 11:42:24.662875 20060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:24.656496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:47633 (local address 127.24.76.66:34301)
0808 11:42:24.656717 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:24.656724 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:24.657498 (+ 774us) server_negotiation.cc:407] Connection header received
0808 11:42:24.657663 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:24.657667 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:24.657733 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:24.657848 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:42:24.658534 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:24.659221 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:24.661152 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:24.661953 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:24.662097 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:24.662169 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:24.662502 (+ 333us) server_negotiation.cc:299] Negotiation successful
0808 11:42:24.662623 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 11:42:24.663030 20083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:24.656415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:34301 (local address 127.24.76.65:47633)
0808 11:42:24.657368 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:24.657386 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:24.657538 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:24.657861 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:24.657869 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:24.657915 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:42:24.658352 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:24.658364 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:24.659353 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:24.659357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:24.660999 (+ 1642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:24.661010 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:24.662205 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:24.662210 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:24.662345 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:24.662381 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 11:42:24.662468 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 11:42:24.662599 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":847,"mutex_wait_us":76,"thread_start_us":526,"threads_started":1}
I20260808 11:42:24.663281 20084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:24.656290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:40355 (local address 127.24.76.65:54193)
0808 11:42:24.657268 (+ 978us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:24.657293 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:24.657424 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:24.657771 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:24.657775 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:24.657800 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:42:24.658277 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:24.658298 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:24.659382 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:24.659386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:24.660994 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:24.661007 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:24.662164 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:24.662170 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:24.662310 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:24.662369 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 11:42:24.662450 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 11:42:24.662573 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":857,"mutex_wait_us":78,"spinlock_wait_cycles":26880,"thread_start_us":671,"threads_started":1}
I20260808 11:42:24.664085 19753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf636e3b0c8f4b7984fa5123c38f7d87" candidate_uuid: "a11ee888e7e2432a8554efbbcc3b7915" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "688a5a7673f64325b25705862a2ee86c" is_pre_election: true
I20260808 11:42:24.664276 19966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf636e3b0c8f4b7984fa5123c38f7d87" candidate_uuid: "a11ee888e7e2432a8554efbbcc3b7915" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43ebcfbe75894bfe9980098f8b1bba5f" is_pre_election: true
I20260808 11:42:24.664665 19753 raft_consensus.cc:2454] T cf636e3b0c8f4b7984fa5123c38f7d87 P 688a5a7673f64325b25705862a2ee86c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a11ee888e7e2432a8554efbbcc3b7915 in term 0.
I20260808 11:42:24.664813 19966 raft_consensus.cc:2454] T cf636e3b0c8f4b7984fa5123c38f7d87 P 43ebcfbe75894bfe9980098f8b1bba5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a11ee888e7e2432a8554efbbcc3b7915 in term 0.
I20260808 11:42:24.665354 19402 leader_election.cc:302] T cf636e3b0c8f4b7984fa5123c38f7d87 P a11ee888e7e2432a8554efbbcc3b7915 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 688a5a7673f64325b25705862a2ee86c, a11ee888e7e2432a8554efbbcc3b7915; no voters:
I20260808 11:42:24.665809 20080 raft_consensus.cc:2781] T cf636e3b0c8f4b7984fa5123c38f7d87 P a11ee888e7e2432a8554efbbcc3b7915 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:42:24.665997 20080 raft_consensus.cc:482] T cf636e3b0c8f4b7984fa5123c38f7d87 P a11ee888e7e2432a8554efbbcc3b7915 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:42:24.666109 20080 raft_consensus.cc:3037] T cf636e3b0c8f4b7984fa5123c38f7d87 P a11ee888e7e2432a8554efbbcc3b7915 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:42:24.668236 20080 raft_consensus.cc:504] T cf636e3b0c8f4b7984fa5123c38f7d87 P a11ee888e7e2432a8554efbbcc3b7915 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a11ee888e7e2432a8554efbbcc3b7915" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 45473 } } peers { permanent_uuid: "688a5a7673f64325b25705862a2ee86c" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 34301 } } peers { permanent_uuid: "43ebcfbe75894bfe9980098f8b1bba5f" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 40355 } }
I20260808 11:42:24.668799 20080 leader_election.cc:288] T cf636e3b0c8f4b7984fa5123c38f7d87 P a11ee888e7e2432a8554efbbcc3b7915 [CANDIDATE]: Term 1 election: Requested vote from peers 688a5a7673f64325b25705862a2ee86c (127.24.76.66:34301), 43ebcfbe75894bfe9980098f8b1bba5f (127.24.76.67:40355)
I20260808 11:42:24.669214 19753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf636e3b0c8f4b7984fa5123c38f7d87" candidate_uuid: "a11ee888e7e2432a8554efbbcc3b7915" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "688a5a7673f64325b25705862a2ee86c"
I20260808 11:42:24.669248 19966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf636e3b0c8f4b7984fa5123c38f7d87" candidate_uuid: "a11ee888e7e2432a8554efbbcc3b7915" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43ebcfbe75894bfe9980098f8b1bba5f"
I20260808 11:42:24.669401 19753 raft_consensus.cc:3037] T cf636e3b0c8f4b7984fa5123c38f7d87 P 688a5a7673f64325b25705862a2ee86c [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:42:24.669456 19966 raft_consensus.cc:3037] T cf636e3b0c8f4b7984fa5123c38f7d87 P 43ebcfbe75894bfe9980098f8b1bba5f [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:42:24.671892 19753 raft_consensus.cc:2454] T cf636e3b0c8f4b7984fa5123c38f7d87 P 688a5a7673f64325b25705862a2ee86c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a11ee888e7e2432a8554efbbcc3b7915 in term 1.
I20260808 11:42:24.672134 19966 raft_consensus.cc:2454] T cf636e3b0c8f4b7984fa5123c38f7d87 P 43ebcfbe75894bfe9980098f8b1bba5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a11ee888e7e2432a8554efbbcc3b7915 in term 1.
I20260808 11:42:24.672263 19402 leader_election.cc:302] T cf636e3b0c8f4b7984fa5123c38f7d87 P a11ee888e7e2432a8554efbbcc3b7915 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 688a5a7673f64325b25705862a2ee86c, a11ee888e7e2432a8554efbbcc3b7915; no voters:
I20260808 11:42:24.672549 20080 raft_consensus.cc:2781] T cf636e3b0c8f4b7984fa5123c38f7d87 P a11ee888e7e2432a8554efbbcc3b7915 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:42:24.672951 20080 raft_consensus.cc:686] T cf636e3b0c8f4b7984fa5123c38f7d87 P a11ee888e7e2432a8554efbbcc3b7915 [term 1 LEADER]: Becoming Leader. State: Replica: a11ee888e7e2432a8554efbbcc3b7915, State: Running, Role: LEADER
I20260808 11:42:24.673398 20080 consensus_queue.cc:237] T cf636e3b0c8f4b7984fa5123c38f7d87 P a11ee888e7e2432a8554efbbcc3b7915 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a11ee888e7e2432a8554efbbcc3b7915" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 45473 } } peers { permanent_uuid: "688a5a7673f64325b25705862a2ee86c" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 34301 } } peers { permanent_uuid: "43ebcfbe75894bfe9980098f8b1bba5f" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 40355 } }
I20260808 11:42:24.678074 19261 catalog_manager.cc:5495] T cf636e3b0c8f4b7984fa5123c38f7d87 P a11ee888e7e2432a8554efbbcc3b7915 reported cstate change: term changed from 0 to 1, leader changed from <none> to a11ee888e7e2432a8554efbbcc3b7915 (127.24.76.65). New cstate: current_term: 1 leader_uuid: "a11ee888e7e2432a8554efbbcc3b7915" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a11ee888e7e2432a8554efbbcc3b7915" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 45473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "688a5a7673f64325b25705862a2ee86c" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 34301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43ebcfbe75894bfe9980098f8b1bba5f" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 40355 } health_report { overall_health: UNKNOWN } } }
I20260808 11:42:24.734579 20058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:24.729127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46276 (local address 127.24.76.65:45473)
0808 11:42:24.729369 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:24.729374 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:24.729394 (+ 20us) server_negotiation.cc:407] Connection header received
0808 11:42:24.729544 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:24.729548 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:24.729640 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:24.729767 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:42:24.730108 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:24.730833 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:24.732395 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:24.732864 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:24.732957 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:24.733462 (+ 505us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:42:24.734045 (+ 583us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:42:24.734165 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:24.734249 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 11:42:24.734318 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 11:42:24.742818 19966 raft_consensus.cc:1261] T cf636e3b0c8f4b7984fa5123c38f7d87 P 43ebcfbe75894bfe9980098f8b1bba5f [term 1 FOLLOWER]: Refusing update from remote peer a11ee888e7e2432a8554efbbcc3b7915: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:42:24.742818 19753 raft_consensus.cc:1261] T cf636e3b0c8f4b7984fa5123c38f7d87 P 688a5a7673f64325b25705862a2ee86c [term 1 FOLLOWER]: Refusing update from remote peer a11ee888e7e2432a8554efbbcc3b7915: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:42:24.743854 20086 consensus_queue.cc:1035] T cf636e3b0c8f4b7984fa5123c38f7d87 P a11ee888e7e2432a8554efbbcc3b7915 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43ebcfbe75894bfe9980098f8b1bba5f" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 40355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:42:24.744196 20080 consensus_queue.cc:1035] T cf636e3b0c8f4b7984fa5123c38f7d87 P a11ee888e7e2432a8554efbbcc3b7915 [LEADER]: Connected to new peer: Peer: permanent_uuid: "688a5a7673f64325b25705862a2ee86c" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 34301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:42:24.755786 20091 mvcc.cc:205] Tried to move back new op lower bound from 7316231556055826432 to 7316231555786022912. Current Snapshot: MvccSnapshot[applied={T|T < 7316231556055826432}]
I20260808 11:42:24.758639 20092 mvcc.cc:205] Tried to move back new op lower bound from 7316231556055826432 to 7316231555786022912. Current Snapshot: MvccSnapshot[applied={T|T < 7316231556055826432}]
I20260808 11:42:24.768062 19260 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:42:24.768388 19260 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:42:24.772284 19260 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 36bf2f52cac34fbbb950cf546b0deea0: Sending DeleteTablet for 3 replicas of tablet cf636e3b0c8f4b7984fa5123c38f7d87
I20260808 11:42:24.773447 19721 tablet_service.cc:1514] Processing DeleteTablet for tablet cf636e3b0c8f4b7984fa5123c38f7d87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:42:24 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:48732
I20260808 11:42:24.773427 19497 tablet_service.cc:1514] Processing DeleteTablet for tablet cf636e3b0c8f4b7984fa5123c38f7d87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:42:24 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:46262
I20260808 11:42:24.773563 19931 tablet_service.cc:1514] Processing DeleteTablet for tablet cf636e3b0c8f4b7984fa5123c38f7d87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:42:24 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:46436
I20260808 11:42:24.775264 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 19371
I20260808 11:42:24.786435 20110 tablet_replica.cc:330] T cf636e3b0c8f4b7984fa5123c38f7d87 P 43ebcfbe75894bfe9980098f8b1bba5f: stopping tablet replica
I20260808 11:42:24.786921 20110 raft_consensus.cc:2229] T cf636e3b0c8f4b7984fa5123c38f7d87 P 43ebcfbe75894bfe9980098f8b1bba5f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:42:24.787268 20108 tablet_replica.cc:330] T cf636e3b0c8f4b7984fa5123c38f7d87 P 688a5a7673f64325b25705862a2ee86c: stopping tablet replica
I20260808 11:42:24.787467 20110 raft_consensus.cc:2258] T cf636e3b0c8f4b7984fa5123c38f7d87 P 43ebcfbe75894bfe9980098f8b1bba5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:42:24.787671 20108 raft_consensus.cc:2229] T cf636e3b0c8f4b7984fa5123c38f7d87 P 688a5a7673f64325b25705862a2ee86c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:42:24.789884 20110 ts_tablet_manager.cc:1905] T cf636e3b0c8f4b7984fa5123c38f7d87 P 43ebcfbe75894bfe9980098f8b1bba5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:42:24.794024 20108 raft_consensus.cc:2258] T cf636e3b0c8f4b7984fa5123c38f7d87 P 688a5a7673f64325b25705862a2ee86c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:42:24.794826 20107 tablet_replica.cc:330] T cf636e3b0c8f4b7984fa5123c38f7d87 P a11ee888e7e2432a8554efbbcc3b7915: stopping tablet replica
I20260808 11:42:24.795221 20107 raft_consensus.cc:2229] T cf636e3b0c8f4b7984fa5123c38f7d87 P a11ee888e7e2432a8554efbbcc3b7915 [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:42:24.795897 20107 raft_consensus.cc:2258] T cf636e3b0c8f4b7984fa5123c38f7d87 P a11ee888e7e2432a8554efbbcc3b7915 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:42:24.798096 20108 ts_tablet_manager.cc:1905] T cf636e3b0c8f4b7984fa5123c38f7d87 P 688a5a7673f64325b25705862a2ee86c: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 08 11:42:24 dist-test-slave-tc2s krb5kdc[19188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.65@KRBTEST.COM
I20260808 11:42:24.801699 20110 ts_tablet_manager.cc:1918] T cf636e3b0c8f4b7984fa5123c38f7d87 P 43ebcfbe75894bfe9980098f8b1bba5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:42:24.801867 20110 log.cc:1196] T cf636e3b0c8f4b7984fa5123c38f7d87 P 43ebcfbe75894bfe9980098f8b1bba5f: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-2/wal/wals/cf636e3b0c8f4b7984fa5123c38f7d87
I20260808 11:42:24.802433 20110 ts_tablet_manager.cc:1939] T cf636e3b0c8f4b7984fa5123c38f7d87 P 43ebcfbe75894bfe9980098f8b1bba5f: Deleting consensus metadata
I20260808 11:42:24.804363 19241 catalog_manager.cc:4841] TS 43ebcfbe75894bfe9980098f8b1bba5f (127.24.76.67:40355): tablet cf636e3b0c8f4b7984fa5123c38f7d87 (table test-table [id=dc238270df134f90b850b72e77797fb3]) successfully deleted
I20260808 11:42:24.806321 20058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:24.776710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46282 (local address 127.24.76.65:45473)
0808 11:42:24.776891 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:24.776899 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:24.786645 (+ 9746us) server_negotiation.cc:407] Connection header received
0808 11:42:24.786821 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:24.786826 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:24.786914 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:24.787029 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:24.789340 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:24.790792 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:24.793260 (+ 2468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:24.794064 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:24.794166 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:24.799334 (+ 5168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:24.799348 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:24.799362 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:42:24.799395 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:24.802750 (+ 3355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:24.803636 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:24.803645 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:24.803649 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:24.803818 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:24.804274 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:24.804283 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:24.804287 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:24.804722 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:24.805044 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:24.805671 (+ 627us) server_negotiation.cc:299] Negotiation successful
0808 11:42:24.805946 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 11:42:24.806615 20107 ts_tablet_manager.cc:1905] T cf636e3b0c8f4b7984fa5123c38f7d87 P a11ee888e7e2432a8554efbbcc3b7915: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:42:24.807734 19598 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:42:24.810362 20108 ts_tablet_manager.cc:1918] T cf636e3b0c8f4b7984fa5123c38f7d87 P 688a5a7673f64325b25705862a2ee86c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:42:24.810570 20108 log.cc:1196] T cf636e3b0c8f4b7984fa5123c38f7d87 P 688a5a7673f64325b25705862a2ee86c: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786189042529715-24881-0/minicluster-data/ts-1/wal/wals/cf636e3b0c8f4b7984fa5123c38f7d87
I20260808 11:42:24.811172 20108 ts_tablet_manager.cc:1939] T cf636e3b0c8f4b7984fa5123c38f7d87 P 688a5a7673f64325b25705862a2ee86c: Deleting consensus metadata
I20260808 11:42:24.812741 19239 catalog_manager.cc:4841] TS 688a5a7673f64325b25705862a2ee86c (127.24.76.66:34301): tablet cf636e3b0c8f4b7984fa5123c38f7d87 (table test-table [id=dc238270df134f90b850b72e77797fb3]) successfully deleted
I20260808 11:42:24.904680 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 19371
W20260808 11:42:24.929585 19238 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.76.65:45473 (error 108)
I20260808 11:42:24.929833 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 19617
I20260808 11:42:24.931281 20061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:24.930809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:45473 (local address 127.0.0.1:46298)
0808 11:42:24.930986 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:24.931107 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:45473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260808 11:42:24.931627 19238 catalog_manager.cc:4584] TS a11ee888e7e2432a8554efbbcc3b7915 (127.24.76.65:45473): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cf636e3b0c8f4b7984fa5123c38f7d87: Network error: Client connection negotiation failed: client connection to 127.24.76.65:45473: connect: Connection refused (error 111)
Aug 08 11:42:24 dist-test-slave-tc2s krb5kdc[19188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.66@KRBTEST.COM
I20260808 11:42:24.944512 20060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:24.930782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48744 (local address 127.24.76.66:34301)
0808 11:42:24.930970 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:24.930976 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:24.930999 (+ 23us) server_negotiation.cc:407] Connection header received
0808 11:42:24.931069 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:24.931074 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:24.931153 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:24.931284 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:24.932763 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:24.933578 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:24.934855 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:24.935246 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:24.935350 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:24.939170 (+ 3820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:24.939184 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:24.939199 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:42:24.939234 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:24.941805 (+ 2571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:24.942533 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:24.942541 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:24.942543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:24.942668 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:24.943029 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:24.943033 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:24.943035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:24.943404 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:24.943685 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:24.944004 (+ 319us) server_negotiation.cc:299] Negotiation successful
0808 11:42:24.944210 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 11:42:24.945168 19813 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:42:24.985287 20061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:24.984812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:45473 (local address 127.0.0.1:46300)
0808 11:42:24.985039 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:24.985141 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:45473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260808 11:42:24.986330 20061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:24.985984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:45473 (local address 127.0.0.1:46314)
0808 11:42:24.986127 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:24.986208 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:45473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 11:42:25.034025 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 19617
I20260808 11:42:25.046833 20061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:25.046355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:45473 (local address 127.0.0.1:46324)
0808 11:42:25.046565 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:25.046660 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:45473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260808 11:42:25.047971 20061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:25.047524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:45473 (local address 127.0.0.1:46338)
0808 11:42:25.047746 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:25.047830 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:45473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260808 11:42:25.058487 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 19827
Aug 08 11:42:25 dist-test-slave-tc2s krb5kdc[19188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.67@KRBTEST.COM
I20260808 11:42:25.072736 20062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:25.059234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46438 (local address 127.24.76.67:40355)
0808 11:42:25.059418 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:25.059423 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:25.059442 (+ 19us) server_negotiation.cc:407] Connection header received
0808 11:42:25.059590 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:25.059594 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:25.059656 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:25.059752 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:25.061112 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:25.061828 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:25.063148 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:25.063543 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:25.063649 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:25.067989 (+ 4340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:25.068004 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:25.068023 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:42:25.068057 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:25.070530 (+ 2473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:25.071148 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:25.071155 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:25.071157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:25.071281 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:25.071554 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:25.071557 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:25.071559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:25.071879 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:25.072104 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:25.072313 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 11:42:25.072467 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 11:42:25.073450 20026 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:42:25.128669 20061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:25.128184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:45473 (local address 127.0.0.1:46348)
0808 11:42:25.128388 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:25.128491 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:45473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260808 11:42:25.129761 20061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:25.129387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:45473 (local address 127.0.0.1:46364)
0808 11:42:25.129579 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:25.129651 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:45473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260808 11:42:25.161247 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 19827
I20260808 11:42:25.184070 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 19214
I20260808 11:42:25.185205 19322 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:42:25.296002 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 19214
2026-08-08T11:42:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4596 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T11:42:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:42:25Z Disabled control of system clock
I20260808 11:42:25.392180 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:39373
--webserver_interface=127.24.76.126
--webserver_port=0
--builtin_ntp_servers=127.24.76.84:38757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:39373
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 11:42:25.725677 20134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:42:25.725965 20134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:42:25.726097 20134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:42:25.733213 20134 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:42:25.733314 20134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:42:25.733355 20134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:42:25.733434 20134 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:42:25.733489 20134 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:42:25.742611 20134 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:38757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:39373
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:39373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:42:25.743916 20134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:42:25.745695 20134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:42:25.756886 20143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:25.756937 20145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:25.756826 20141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:42:25.757829 20134 server_base.cc:1034] running on GCE node
I20260808 11:42:25.758607 20134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:42:25.759908 20134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:42:25.761103 20134 hybrid_clock.cc:648] HybridClock initialized: now 1786189345761045 us; error 54 us; skew 500 ppm
I20260808 11:42:25.761404 20134 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:42:25.763574 20134 webserver.cc:460] Webserver started at http://127.24.76.126:40603/ using document root <none> and password file <none>
I20260808 11:42:25.764212 20134 fs_manager.cc:359] Metadata directory not provided
I20260808 11:42:25.764298 20134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:42:25.764564 20134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:42:25.767429 20134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "67ef9188482d4a94a509f4e30df7e7b1"
format_stamp: "Formatted at 2026-08-08 11:42:25 on dist-test-slave-tc2s"
server_key: "0f973956e564862288bd155e21c92ae5"
server_key_iv: "050727cda72aa5637046df4c991276b8"
server_key_version: "encryptionkey@0"
I20260808 11:42:25.768183 20134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "67ef9188482d4a94a509f4e30df7e7b1"
format_stamp: "Formatted at 2026-08-08 11:42:25 on dist-test-slave-tc2s"
server_key: "0f973956e564862288bd155e21c92ae5"
server_key_iv: "050727cda72aa5637046df4c991276b8"
server_key_version: "encryptionkey@0"
I20260808 11:42:25.772990 20134 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 11:42:25.776310 20158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:25.777539 20134 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 11:42:25.777684 20134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "67ef9188482d4a94a509f4e30df7e7b1"
format_stamp: "Formatted at 2026-08-08 11:42:25 on dist-test-slave-tc2s"
server_key: "0f973956e564862288bd155e21c92ae5"
server_key_iv: "050727cda72aa5637046df4c991276b8"
server_key_version: "encryptionkey@0"
I20260808 11:42:25.777854 20134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:42:25.791734 20134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:42:25.792454 20134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:42:25.792685 20134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:42:25.810240 20134 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:39373
I20260808 11:42:25.810248 20244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:39373 every 8 connection(s)
I20260808 11:42:25.811560 20134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:42:25.815277 20245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:42:25.818194 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 20134
I20260808 11:42:25.818342 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:42:25.818801 24881 external_mini_cluster.cc:1428] Setting key 25bd137ccf4eac08a2973f740be300cf
I20260808 11:42:25.827062 20245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67ef9188482d4a94a509f4e30df7e7b1: Bootstrap starting.
I20260808 11:42:25.830704 20245 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67ef9188482d4a94a509f4e30df7e7b1: Neither blocks nor log segments found. Creating new log.
I20260808 11:42:25.831903 20245 log.cc:824] T 00000000000000000000000000000000 P 67ef9188482d4a94a509f4e30df7e7b1: Log is configured to *not* fsync() on all Append() calls
I20260808 11:42:25.832610 20249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:25.820613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47820 (local address 127.24.76.126:39373)
0808 11:42:25.821522 (+ 909us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:25.821532 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:25.821574 (+ 42us) server_negotiation.cc:407] Connection header received
0808 11:42:25.823282 (+ 1708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:25.823308 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:25.823761 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:25.824085 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:25.824701 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:25.826024 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:25.827919 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:25.828625 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:25.828765 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:25.830052 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:25.830060 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:25.830074 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:42:25.830694 (+ 620us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:25.830794 (+ 100us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:42:25.830825 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:25.830950 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:25.831190 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 11:42:25.831496 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":345,"threads_started":1}
I20260808 11:42:25.835194 20245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67ef9188482d4a94a509f4e30df7e7b1: No bootstrap required, opened a new log
I20260808 11:42:25.843108 20245 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67ef9188482d4a94a509f4e30df7e7b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67ef9188482d4a94a509f4e30df7e7b1" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39373 } }
I20260808 11:42:25.843521 20245 raft_consensus.cc:374] T 00000000000000000000000000000000 P 67ef9188482d4a94a509f4e30df7e7b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:42:25.843623 20245 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67ef9188482d4a94a509f4e30df7e7b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67ef9188482d4a94a509f4e30df7e7b1, State: Initialized, Role: FOLLOWER
I20260808 11:42:25.844187 20245 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67ef9188482d4a94a509f4e30df7e7b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67ef9188482d4a94a509f4e30df7e7b1" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39373 } }
I20260808 11:42:25.844425 20245 raft_consensus.cc:388] T 00000000000000000000000000000000 P 67ef9188482d4a94a509f4e30df7e7b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:42:25.844558 20245 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67ef9188482d4a94a509f4e30df7e7b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:42:25.844717 20245 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67ef9188482d4a94a509f4e30df7e7b1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:42:25.847122 20245 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67ef9188482d4a94a509f4e30df7e7b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67ef9188482d4a94a509f4e30df7e7b1" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39373 } }
I20260808 11:42:25.847671 20245 leader_election.cc:302] T 00000000000000000000000000000000 P 67ef9188482d4a94a509f4e30df7e7b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67ef9188482d4a94a509f4e30df7e7b1; no voters:
I20260808 11:42:25.848250 20245 leader_election.cc:288] T 00000000000000000000000000000000 P 67ef9188482d4a94a509f4e30df7e7b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:42:25.848387 20254 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67ef9188482d4a94a509f4e30df7e7b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:42:25.849313 20254 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67ef9188482d4a94a509f4e30df7e7b1 [term 1 LEADER]: Becoming Leader. State: Replica: 67ef9188482d4a94a509f4e30df7e7b1, State: Running, Role: LEADER
I20260808 11:42:25.849798 20254 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67ef9188482d4a94a509f4e30df7e7b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67ef9188482d4a94a509f4e30df7e7b1" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39373 } }
I20260808 11:42:25.850306 20245 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67ef9188482d4a94a509f4e30df7e7b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:42:25.852873 20258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67ef9188482d4a94a509f4e30df7e7b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67ef9188482d4a94a509f4e30df7e7b1. Latest consensus state: current_term: 1 leader_uuid: "67ef9188482d4a94a509f4e30df7e7b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67ef9188482d4a94a509f4e30df7e7b1" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39373 } } }
I20260808 11:42:25.853379 20258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67ef9188482d4a94a509f4e30df7e7b1 [sys.catalog]: This master's current role is: LEADER
I20260808 11:42:25.854370 20256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67ef9188482d4a94a509f4e30df7e7b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "67ef9188482d4a94a509f4e30df7e7b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67ef9188482d4a94a509f4e30df7e7b1" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39373 } } }
I20260808 11:42:25.854717 20256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67ef9188482d4a94a509f4e30df7e7b1 [sys.catalog]: This master's current role is: LEADER
I20260808 11:42:25.855779 20270 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:42:25.861228 20270 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:42:25.869513 20270 catalog_manager.cc:1347] Generated new cluster ID: cef9899dadee4ec1b5161af286b29f21
I20260808 11:42:25.869622 20270 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:42:25.895977 20270 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:42:25.897799 20270 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:42:25.904295 20270 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67ef9188482d4a94a509f4e30df7e7b1: Generated new TSK 0
I20260808 11:42:25.905088 20270 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:42:25.915825 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.65:0
--local_ip_for_outbound_sockets=127.24.76.65
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:39373
--builtin_ntp_servers=127.24.76.84:38757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 11:42:26.272532 20290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:42:26.272800 20290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:42:26.272891 20290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:42:26.279955 20290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:42:26.280052 20290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:42:26.280165 20290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.65
I20260808 11:42:26.289175 20290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:38757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:39373
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.65
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:42:26.290541 20290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:42:26.292248 20290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:42:26.304946 20302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:26.305038 20296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:26.305280 20297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:42:26.306052 20290 server_base.cc:1034] running on GCE node
I20260808 11:42:26.306654 20290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:42:26.307548 20290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:42:26.308761 20290 hybrid_clock.cc:648] HybridClock initialized: now 1786189346308706 us; error 49 us; skew 500 ppm
I20260808 11:42:26.309072 20290 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:42:26.311319 20290 webserver.cc:460] Webserver started at http://127.24.76.65:40987/ using document root <none> and password file <none>
I20260808 11:42:26.312012 20290 fs_manager.cc:359] Metadata directory not provided
I20260808 11:42:26.312124 20290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:42:26.312414 20290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:42:26.315274 20290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-0/data/instance:
uuid: "097ced8c992f4403968cf854115f981a"
format_stamp: "Formatted at 2026-08-08 11:42:26 on dist-test-slave-tc2s"
server_key: "7a4a6d9375ad21c7cc6f3e0bd8857291"
server_key_iv: "255b8504648ccabda7b61dd213e1a744"
server_key_version: "encryptionkey@0"
I20260808 11:42:26.316038 20290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance:
uuid: "097ced8c992f4403968cf854115f981a"
format_stamp: "Formatted at 2026-08-08 11:42:26 on dist-test-slave-tc2s"
server_key: "7a4a6d9375ad21c7cc6f3e0bd8857291"
server_key_iv: "255b8504648ccabda7b61dd213e1a744"
server_key_version: "encryptionkey@0"
I20260808 11:42:26.320902 20290 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 11:42:26.324213 20310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:26.325421 20290 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 11:42:26.325577 20290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-0/wal
uuid: "097ced8c992f4403968cf854115f981a"
format_stamp: "Formatted at 2026-08-08 11:42:26 on dist-test-slave-tc2s"
server_key: "7a4a6d9375ad21c7cc6f3e0bd8857291"
server_key_iv: "255b8504648ccabda7b61dd213e1a744"
server_key_version: "encryptionkey@0"
I20260808 11:42:26.325764 20290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:42:26.337858 20290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:42:26.338706 20290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:42:26.339001 20290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:42:26.339906 20290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:42:26.341307 20290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:42:26.341403 20290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:26.341513 20290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:42:26.341581 20290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:26.371979 20290 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.65:37931
I20260808 11:42:26.372013 20492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.65:37931 every 8 connection(s)
I20260808 11:42:26.373387 20290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
I20260808 11:42:26.381722 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 20290
I20260808 11:42:26.381873 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance
I20260808 11:42:26.382315 24881 external_mini_cluster.cc:1428] Setting key 506047b95f870bede6451421f2af58bb
I20260808 11:42:26.385105 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.66:0
--local_ip_for_outbound_sockets=127.24.76.66
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:39373
--builtin_ntp_servers=127.24.76.84:38757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 11:42:26.386046 20496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:26.375957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:56015 (local address 127.24.76.126:39373)
0808 11:42:26.376448 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:26.376453 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:26.377582 (+ 1129us) server_negotiation.cc:407] Connection header received
0808 11:42:26.379392 (+ 1810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:26.379397 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:26.379484 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:26.379581 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:26.380635 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:26.381364 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:26.383207 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:26.383665 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:26.383762 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:26.385229 (+ 1467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:26.385235 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:26.385239 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:42:26.385261 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:26.385312 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:42:26.385330 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:26.385414 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:26.385602 (+ 188us) server_negotiation.cc:299] Negotiation successful
0808 11:42:26.385691 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":221,"threads_started":1}
I20260808 11:42:26.386893 20497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:26.376527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:39373 (local address 127.24.76.65:56015)
0808 11:42:26.377415 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:26.377460 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:26.379122 (+ 1662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:26.379819 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:26.379827 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:26.379910 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:42:26.380473 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:26.380490 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:26.381528 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:26.381532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:26.383081 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:26.383097 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:26.383892 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:26.383898 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:26.384229 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:26.384910 (+ 681us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:42:26.384912 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:42:26.384950 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:42:26.385043 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:42:26.385421 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:42:26.385431 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:42:26.385437 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 11:42:26.385573 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 11:42:26.385846 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":283,"threads_started":1}
I20260808 11:42:26.388280 20494 heartbeater.cc:347] Connected to a master server at 127.24.76.126:39373
I20260808 11:42:26.388556 20494 heartbeater.cc:464] Registering TS with master...
I20260808 11:42:26.389247 20494 heartbeater.cc:511] Master 127.24.76.126:39373 requested a full tablet report, sending...
I20260808 11:42:26.391207 20188 ts_manager.cc:194] Registered new tserver with Master: 097ced8c992f4403968cf854115f981a (127.24.76.65:37931)
I20260808 11:42:26.392675 20188 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.76.65:56015
W20260808 11:42:26.735044 20498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:42:26.735325 20498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:42:26.735464 20498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:42:26.742640 20498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:42:26.742746 20498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:42:26.742904 20498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.66
I20260808 11:42:26.751961 20498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:38757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:39373
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.66
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:42:26.753227 20498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:42:26.755007 20498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:42:26.767349 20506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:26.767434 20507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:26.767475 20510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:42:26.768146 20498 server_base.cc:1034] running on GCE node
I20260808 11:42:26.768718 20498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:42:26.769584 20498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:42:26.770792 20498 hybrid_clock.cc:648] HybridClock initialized: now 1786189346770737 us; error 45 us; skew 500 ppm
I20260808 11:42:26.771086 20498 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:42:26.773164 20498 webserver.cc:460] Webserver started at http://127.24.76.66:40017/ using document root <none> and password file <none>
I20260808 11:42:26.773798 20498 fs_manager.cc:359] Metadata directory not provided
I20260808 11:42:26.773921 20498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:42:26.774219 20498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:42:26.776971 20498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-1/data/instance:
uuid: "bbad78bf66044c14beead1caf3caf528"
format_stamp: "Formatted at 2026-08-08 11:42:26 on dist-test-slave-tc2s"
server_key: "fc80b5068efa4a1ff81414be9adc38d5"
server_key_iv: "f849858a8df9dbc7f0bcf969058bf28a"
server_key_version: "encryptionkey@0"
I20260808 11:42:26.777724 20498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance:
uuid: "bbad78bf66044c14beead1caf3caf528"
format_stamp: "Formatted at 2026-08-08 11:42:26 on dist-test-slave-tc2s"
server_key: "fc80b5068efa4a1ff81414be9adc38d5"
server_key_iv: "f849858a8df9dbc7f0bcf969058bf28a"
server_key_version: "encryptionkey@0"
I20260808 11:42:26.782747 20498 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260808 11:42:26.785955 20516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:26.787170 20498 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 11:42:26.787318 20498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-1/wal
uuid: "bbad78bf66044c14beead1caf3caf528"
format_stamp: "Formatted at 2026-08-08 11:42:26 on dist-test-slave-tc2s"
server_key: "fc80b5068efa4a1ff81414be9adc38d5"
server_key_iv: "f849858a8df9dbc7f0bcf969058bf28a"
server_key_version: "encryptionkey@0"
I20260808 11:42:26.787539 20498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:42:26.802836 20498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:42:26.803560 20498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:42:26.803833 20498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:42:26.804718 20498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:42:26.806142 20498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:42:26.806238 20498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:26.806375 20498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:42:26.806450 20498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:26.840080 20498 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.66:46529
I20260808 11:42:26.840124 20711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.66:46529 every 8 connection(s)
I20260808 11:42:26.841391 20498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
I20260808 11:42:26.851486 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 20498
I20260808 11:42:26.851673 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance
I20260808 11:42:26.852138 24881 external_mini_cluster.cc:1428] Setting key d6aa9f2ca4d06035d23e3e94b0f612ff
I20260808 11:42:26.853695 20496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:26.843897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:57961 (local address 127.24.76.126:39373)
0808 11:42:26.844090 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:26.844095 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:26.845506 (+ 1411us) server_negotiation.cc:407] Connection header received
0808 11:42:26.847300 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:26.847305 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:26.847382 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:26.847482 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:26.848574 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:26.849287 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:26.850884 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:26.851294 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:26.851405 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:26.852865 (+ 1460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:26.852872 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:26.852877 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:42:26.852899 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:26.852940 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:42:26.852959 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:26.853046 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:26.853265 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 11:42:26.853366 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 11:42:26.854975 20716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:26.844447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:39373 (local address 127.24.76.66:57961)
0808 11:42:26.845331 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:26.845367 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:26.847025 (+ 1658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:26.847717 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:26.847725 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:26.847810 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:42:26.848400 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:26.848419 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:26.849440 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:26.849444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:26.850724 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:26.850736 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:26.851512 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:26.851519 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:26.851792 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:26.852494 (+ 702us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:42:26.852498 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:42:26.852560 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:42:26.852657 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:42:26.853061 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:42:26.853073 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:42:26.853082 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 11:42:26.853236 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 11:42:26.853619 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":257,"threads_started":1}
I20260808 11:42:26.855356 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.67:0
--local_ip_for_outbound_sockets=127.24.76.67
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:39373
--builtin_ntp_servers=127.24.76.84:38757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 11:42:26.856383 20712 heartbeater.cc:347] Connected to a master server at 127.24.76.126:39373
I20260808 11:42:26.856714 20712 heartbeater.cc:464] Registering TS with master...
I20260808 11:42:26.857594 20712 heartbeater.cc:511] Master 127.24.76.126:39373 requested a full tablet report, sending...
I20260808 11:42:26.859022 20188 ts_manager.cc:194] Registered new tserver with Master: bbad78bf66044c14beead1caf3caf528 (127.24.76.66:46529)
I20260808 11:42:26.859790 20188 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.76.66:57961
W20260808 11:42:27.195993 20718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:42:27.196262 20718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:42:27.196384 20718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:42:27.203509 20718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:42:27.203600 20718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:42:27.203733 20718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.67
I20260808 11:42:27.212527 20718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:38757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:39373
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.67
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:42:27.213752 20718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:42:27.215562 20718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:42:27.227936 20724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:27.228013 20725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:27.228032 20728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:42:27.229066 20718 server_base.cc:1034] running on GCE node
I20260808 11:42:27.229635 20718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:42:27.230647 20718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:42:27.231839 20718 hybrid_clock.cc:648] HybridClock initialized: now 1786189347231791 us; error 42 us; skew 500 ppm
I20260808 11:42:27.232147 20718 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:42:27.234290 20718 webserver.cc:460] Webserver started at http://127.24.76.67:44689/ using document root <none> and password file <none>
I20260808 11:42:27.234939 20718 fs_manager.cc:359] Metadata directory not provided
I20260808 11:42:27.235042 20718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:42:27.235329 20718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:42:27.238123 20718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-2/data/instance:
uuid: "9b5d85efc1b14ff0a9f67ea5af63ad1c"
format_stamp: "Formatted at 2026-08-08 11:42:27 on dist-test-slave-tc2s"
server_key: "11ccd548c22ba769c6701f0db2d3f61b"
server_key_iv: "baf0885fe0f6950a12cfa1d9f215f2a6"
server_key_version: "encryptionkey@0"
I20260808 11:42:27.238885 20718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance:
uuid: "9b5d85efc1b14ff0a9f67ea5af63ad1c"
format_stamp: "Formatted at 2026-08-08 11:42:27 on dist-test-slave-tc2s"
server_key: "11ccd548c22ba769c6701f0db2d3f61b"
server_key_iv: "baf0885fe0f6950a12cfa1d9f215f2a6"
server_key_version: "encryptionkey@0"
I20260808 11:42:27.243695 20718 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 11:42:27.247071 20738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:27.248279 20718 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.004s sys 0.000s
I20260808 11:42:27.248426 20718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-2/wal
uuid: "9b5d85efc1b14ff0a9f67ea5af63ad1c"
format_stamp: "Formatted at 2026-08-08 11:42:27 on dist-test-slave-tc2s"
server_key: "11ccd548c22ba769c6701f0db2d3f61b"
server_key_iv: "baf0885fe0f6950a12cfa1d9f215f2a6"
server_key_version: "encryptionkey@0"
I20260808 11:42:27.248615 20718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:42:27.279542 20718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:42:27.280364 20718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:42:27.280642 20718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:42:27.281659 20718 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:42:27.283200 20718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:42:27.283301 20718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:27.283403 20718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:42:27.283471 20718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:27.313800 20718 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.67:34895
I20260808 11:42:27.313915 20921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.67:34895 every 8 connection(s)
I20260808 11:42:27.315141 20718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
I20260808 11:42:27.315711 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 20718
I20260808 11:42:27.315874 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance
I20260808 11:42:27.316352 24881 external_mini_cluster.cc:1428] Setting key 3be6ff62e8018d43ec5a352798f9dc31
I20260808 11:42:27.327608 20496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:27.317951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:47551 (local address 127.24.76.126:39373)
0808 11:42:27.318109 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:27.318113 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:27.319720 (+ 1607us) server_negotiation.cc:407] Connection header received
0808 11:42:27.321505 (+ 1785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:27.321510 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:27.321589 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:27.321672 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:27.322783 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:27.323525 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:27.325029 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:27.325421 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:27.325529 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:27.326827 (+ 1298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:27.326832 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:27.326835 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:42:27.326852 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:27.326884 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:42:27.326896 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:27.326971 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:27.327191 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 11:42:27.327287 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 11:42:27.328514 20924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:27.318548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:39373 (local address 127.24.76.67:47551)
0808 11:42:27.319573 (+ 1025us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:27.319620 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:27.321263 (+ 1643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:27.321945 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:27.321958 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:27.322045 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:42:27.322626 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:27.322644 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:27.323672 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:27.323676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:27.324884 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:27.324896 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:27.325642 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:27.325650 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:27.325852 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:27.326538 (+ 686us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:42:27.326540 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:42:27.326577 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:42:27.326648 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:42:27.326983 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:42:27.326995 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:42:27.327003 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 11:42:27.327146 (+ 143us) client_negotiation.cc:240] Negotiation successful
0808 11:42:27.327515 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":323,"threads_started":1}
I20260808 11:42:27.329694 20922 heartbeater.cc:347] Connected to a master server at 127.24.76.126:39373
I20260808 11:42:27.330058 20922 heartbeater.cc:464] Registering TS with master...
I20260808 11:42:27.330932 20922 heartbeater.cc:511] Master 127.24.76.126:39373 requested a full tablet report, sending...
I20260808 11:42:27.332388 20188 ts_manager.cc:194] Registered new tserver with Master: 9b5d85efc1b14ff0a9f67ea5af63ad1c (127.24.76.67:34895)
I20260808 11:42:27.333133 20188 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.76.67:47551
I20260808 11:42:27.343729 24881 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:42:27.353564 20496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:27.347689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47826 (local address 127.24.76.126:39373)
0808 11:42:27.347890 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:27.347897 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:27.348283 (+ 386us) server_negotiation.cc:407] Connection header received
0808 11:42:27.348444 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:27.348449 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:27.348534 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:27.348617 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:27.349087 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:27.349968 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:27.351340 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:27.351769 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:27.351876 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:27.352895 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:27.352900 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:27.352904 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:42:27.352920 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:27.352958 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:42:27.352980 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:27.353055 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:27.353192 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 11:42:27.353282 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 11:42:27.357350 20188 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47826:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:42:27.363998 20188 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:42:27.395517 20939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:27.375313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:37931 (local address 127.0.0.1:43206)
0808 11:42:27.387937 (+ 12624us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:27.388007 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:27.388210 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:27.389928 (+ 1718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:27.389936 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:27.389966 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:42:27.390336 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:27.390348 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:27.391764 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:27.391770 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:27.393277 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:27.393291 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:27.394624 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:27.394631 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:27.394842 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:27.394897 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 11:42:27.395011 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 11:42:27.395203 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12339,"thread_start_us":13077,"threads_started":1}
I20260808 11:42:27.395448 20494 heartbeater.cc:503] Master 127.24.76.126:39373 was elected leader, sending a full tablet report...
I20260808 11:42:27.395915 20943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:27.375931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43206 (local address 127.24.76.65:37931)
0808 11:42:27.389414 (+ 13483us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:27.389421 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:27.389447 (+ 26us) server_negotiation.cc:407] Connection header received
0808 11:42:27.389529 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:27.389534 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:27.389703 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:27.389878 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:42:27.390512 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:27.391570 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:27.393460 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:27.394422 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:27.394550 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:27.394837 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:27.395075 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 11:42:27.395232 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13331,"thread_start_us":13215,"threads_started":1}
I20260808 11:42:27.400483 20386 tablet_service.cc:1467] Processing CreateTablet for tablet 2dacdfabc455485687a94affd8fa3a45 (DEFAULT_TABLE table=test-table [id=75bcdcc9e2654119820c21a856e661dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:42:27.402143 20386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dacdfabc455485687a94affd8fa3a45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:42:27.410701 20939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:27.375633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:34895 (local address 127.0.0.1:43386)
0808 11:42:27.396151 (+ 20518us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:27.396171 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:27.396278 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:27.403921 (+ 7643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:27.403926 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:27.403952 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:42:27.404135 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:27.404148 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:27.406317 (+ 2169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:27.406332 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:27.408129 (+ 1797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:27.408144 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:27.409929 (+ 1785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:27.409936 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:27.410126 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:27.410160 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 11:42:27.410225 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 11:42:27.410372 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20270,"thread_start_us":28028,"threads_started":1}
I20260808 11:42:27.411804 20945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:27.376269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43386 (local address 127.24.76.67:34895)
0808 11:42:27.402884 (+ 26615us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:27.402901 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:27.402936 (+ 35us) server_negotiation.cc:407] Connection header received
0808 11:42:27.403027 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:27.403033 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:27.403255 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:27.404318 (+ 1063us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:42:27.404597 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:27.406128 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:27.408360 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:27.409726 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:27.410926 (+ 1200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:27.411206 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:27.411277 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 11:42:27.411450 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26450,"thread_start_us":26267,"threads_started":1}
I20260808 11:42:27.414026 20944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:27.376020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:46529 (local address 127.0.0.1:40658)
0808 11:42:27.402551 (+ 26531us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:27.402570 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:27.402700 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:27.404826 (+ 2126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:27.404831 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:27.404894 (+ 63us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:42:27.405410 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:27.405432 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:27.408750 (+ 3318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:27.408756 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:27.410790 (+ 2034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:27.410805 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:27.413209 (+ 2404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:27.413217 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:27.413452 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:27.413486 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 11:42:27.413572 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 11:42:27.413692 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26369,"thread_start_us":26118,"threads_started":1}
I20260808 11:42:27.414804 20947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:27.376598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40658 (local address 127.24.76.66:46529)
0808 11:42:27.401975 (+ 25377us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:27.401984 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:27.402830 (+ 846us) server_negotiation.cc:407] Connection header received
0808 11:42:27.402959 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:27.402968 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:27.403231 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:27.404460 (+ 1229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:42:27.405685 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:27.408587 (+ 2902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:27.411024 (+ 2437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:27.412975 (+ 1951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:27.413104 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:27.413329 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:27.414309 (+ 980us) server_negotiation.cc:299] Negotiation successful
0808 11:42:27.414474 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25139,"thread_start_us":24821,"threads_started":1}
I20260808 11:42:27.415596 20822 tablet_service.cc:1467] Processing CreateTablet for tablet 2dacdfabc455485687a94affd8fa3a45 (DEFAULT_TABLE table=test-table [id=75bcdcc9e2654119820c21a856e661dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:42:27.417151 20822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dacdfabc455485687a94affd8fa3a45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:42:27.418435 20948 tablet_bootstrap.cc:492] T 2dacdfabc455485687a94affd8fa3a45 P 097ced8c992f4403968cf854115f981a: Bootstrap starting.
I20260808 11:42:27.418316 20602 tablet_service.cc:1467] Processing CreateTablet for tablet 2dacdfabc455485687a94affd8fa3a45 (DEFAULT_TABLE table=test-table [id=75bcdcc9e2654119820c21a856e661dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:42:27.419697 20602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dacdfabc455485687a94affd8fa3a45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:42:27.422133 20948 tablet_bootstrap.cc:654] T 2dacdfabc455485687a94affd8fa3a45 P 097ced8c992f4403968cf854115f981a: Neither blocks nor log segments found. Creating new log.
I20260808 11:42:27.423458 20948 log.cc:824] T 2dacdfabc455485687a94affd8fa3a45 P 097ced8c992f4403968cf854115f981a: Log is configured to *not* fsync() on all Append() calls
I20260808 11:42:27.426146 20948 tablet_bootstrap.cc:492] T 2dacdfabc455485687a94affd8fa3a45 P 097ced8c992f4403968cf854115f981a: No bootstrap required, opened a new log
I20260808 11:42:27.426399 20948 ts_tablet_manager.cc:1397] T 2dacdfabc455485687a94affd8fa3a45 P 097ced8c992f4403968cf854115f981a: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20260808 11:42:27.431703 20950 tablet_bootstrap.cc:492] T 2dacdfabc455485687a94affd8fa3a45 P 9b5d85efc1b14ff0a9f67ea5af63ad1c: Bootstrap starting.
I20260808 11:42:27.433637 20953 tablet_bootstrap.cc:492] T 2dacdfabc455485687a94affd8fa3a45 P bbad78bf66044c14beead1caf3caf528: Bootstrap starting.
I20260808 11:42:27.435580 20950 tablet_bootstrap.cc:654] T 2dacdfabc455485687a94affd8fa3a45 P 9b5d85efc1b14ff0a9f67ea5af63ad1c: Neither blocks nor log segments found. Creating new log.
I20260808 11:42:27.435410 20948 raft_consensus.cc:348] T 2dacdfabc455485687a94affd8fa3a45 P 097ced8c992f4403968cf854115f981a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "097ced8c992f4403968cf854115f981a" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 37931 } } peers { permanent_uuid: "9b5d85efc1b14ff0a9f67ea5af63ad1c" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 34895 } } peers { permanent_uuid: "bbad78bf66044c14beead1caf3caf528" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 46529 } }
I20260808 11:42:27.435877 20948 raft_consensus.cc:374] T 2dacdfabc455485687a94affd8fa3a45 P 097ced8c992f4403968cf854115f981a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:42:27.435987 20948 raft_consensus.cc:729] T 2dacdfabc455485687a94affd8fa3a45 P 097ced8c992f4403968cf854115f981a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 097ced8c992f4403968cf854115f981a, State: Initialized, Role: FOLLOWER
I20260808 11:42:27.436594 20953 tablet_bootstrap.cc:654] T 2dacdfabc455485687a94affd8fa3a45 P bbad78bf66044c14beead1caf3caf528: Neither blocks nor log segments found. Creating new log.
I20260808 11:42:27.436671 20948 consensus_queue.cc:260] T 2dacdfabc455485687a94affd8fa3a45 P 097ced8c992f4403968cf854115f981a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "097ced8c992f4403968cf854115f981a" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 37931 } } peers { permanent_uuid: "9b5d85efc1b14ff0a9f67ea5af63ad1c" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 34895 } } peers { permanent_uuid: "bbad78bf66044c14beead1caf3caf528" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 46529 } }
I20260808 11:42:27.437078 20950 log.cc:824] T 2dacdfabc455485687a94affd8fa3a45 P 9b5d85efc1b14ff0a9f67ea5af63ad1c: Log is configured to *not* fsync() on all Append() calls
I20260808 11:42:27.437638 20953 log.cc:824] T 2dacdfabc455485687a94affd8fa3a45 P bbad78bf66044c14beead1caf3caf528: Log is configured to *not* fsync() on all Append() calls
I20260808 11:42:27.439250 20948 ts_tablet_manager.cc:1428] T 2dacdfabc455485687a94affd8fa3a45 P 097ced8c992f4403968cf854115f981a: Time spent starting tablet: real 0.013s user 0.009s sys 0.003s
I20260808 11:42:27.441615 20950 tablet_bootstrap.cc:492] T 2dacdfabc455485687a94affd8fa3a45 P 9b5d85efc1b14ff0a9f67ea5af63ad1c: No bootstrap required, opened a new log
I20260808 11:42:27.441865 20950 ts_tablet_manager.cc:1397] T 2dacdfabc455485687a94affd8fa3a45 P 9b5d85efc1b14ff0a9f67ea5af63ad1c: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.001s
I20260808 11:42:27.442164 20953 tablet_bootstrap.cc:492] T 2dacdfabc455485687a94affd8fa3a45 P bbad78bf66044c14beead1caf3caf528: No bootstrap required, opened a new log
I20260808 11:42:27.442438 20953 ts_tablet_manager.cc:1397] T 2dacdfabc455485687a94affd8fa3a45 P bbad78bf66044c14beead1caf3caf528: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260808 11:42:27.449707 20950 raft_consensus.cc:348] T 2dacdfabc455485687a94affd8fa3a45 P 9b5d85efc1b14ff0a9f67ea5af63ad1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "097ced8c992f4403968cf854115f981a" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 37931 } } peers { permanent_uuid: "9b5d85efc1b14ff0a9f67ea5af63ad1c" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 34895 } } peers { permanent_uuid: "bbad78bf66044c14beead1caf3caf528" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 46529 } }
I20260808 11:42:27.450135 20950 raft_consensus.cc:374] T 2dacdfabc455485687a94affd8fa3a45 P 9b5d85efc1b14ff0a9f67ea5af63ad1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:42:27.450237 20950 raft_consensus.cc:729] T 2dacdfabc455485687a94affd8fa3a45 P 9b5d85efc1b14ff0a9f67ea5af63ad1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b5d85efc1b14ff0a9f67ea5af63ad1c, State: Initialized, Role: FOLLOWER
I20260808 11:42:27.450049 20953 raft_consensus.cc:348] T 2dacdfabc455485687a94affd8fa3a45 P bbad78bf66044c14beead1caf3caf528 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "097ced8c992f4403968cf854115f981a" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 37931 } } peers { permanent_uuid: "9b5d85efc1b14ff0a9f67ea5af63ad1c" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 34895 } } peers { permanent_uuid: "bbad78bf66044c14beead1caf3caf528" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 46529 } }
I20260808 11:42:27.450450 20953 raft_consensus.cc:374] T 2dacdfabc455485687a94affd8fa3a45 P bbad78bf66044c14beead1caf3caf528 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:42:27.450548 20953 raft_consensus.cc:729] T 2dacdfabc455485687a94affd8fa3a45 P bbad78bf66044c14beead1caf3caf528 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbad78bf66044c14beead1caf3caf528, State: Initialized, Role: FOLLOWER
I20260808 11:42:27.450837 20950 consensus_queue.cc:260] T 2dacdfabc455485687a94affd8fa3a45 P 9b5d85efc1b14ff0a9f67ea5af63ad1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "097ced8c992f4403968cf854115f981a" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 37931 } } peers { permanent_uuid: "9b5d85efc1b14ff0a9f67ea5af63ad1c" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 34895 } } peers { permanent_uuid: "bbad78bf66044c14beead1caf3caf528" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 46529 } }
I20260808 11:42:27.451050 20953 consensus_queue.cc:260] T 2dacdfabc455485687a94affd8fa3a45 P bbad78bf66044c14beead1caf3caf528 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "097ced8c992f4403968cf854115f981a" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 37931 } } peers { permanent_uuid: "9b5d85efc1b14ff0a9f67ea5af63ad1c" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 34895 } } peers { permanent_uuid: "bbad78bf66044c14beead1caf3caf528" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 46529 } }
I20260808 11:42:27.451915 20922 heartbeater.cc:503] Master 127.24.76.126:39373 was elected leader, sending a full tablet report...
I20260808 11:42:27.452466 20950 ts_tablet_manager.cc:1428] T 2dacdfabc455485687a94affd8fa3a45 P 9b5d85efc1b14ff0a9f67ea5af63ad1c: Time spent starting tablet: real 0.010s user 0.001s sys 0.008s
I20260808 11:42:27.452771 20953 ts_tablet_manager.cc:1428] T 2dacdfabc455485687a94affd8fa3a45 P bbad78bf66044c14beead1caf3caf528: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260808 11:42:27.453084 20712 heartbeater.cc:503] Master 127.24.76.126:39373 was elected leader, sending a full tablet report...
W20260808 11:42:27.567785 20923 tablet.cc:2369] T 2dacdfabc455485687a94affd8fa3a45 P 9b5d85efc1b14ff0a9f67ea5af63ad1c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:42:27.581490 20954 raft_consensus.cc:482] T 2dacdfabc455485687a94affd8fa3a45 P 097ced8c992f4403968cf854115f981a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:42:27.581801 20954 raft_consensus.cc:504] T 2dacdfabc455485687a94affd8fa3a45 P 097ced8c992f4403968cf854115f981a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "097ced8c992f4403968cf854115f981a" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 37931 } } peers { permanent_uuid: "9b5d85efc1b14ff0a9f67ea5af63ad1c" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 34895 } } peers { permanent_uuid: "bbad78bf66044c14beead1caf3caf528" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 46529 } }
I20260808 11:42:27.583637 20954 leader_election.cc:288] T 2dacdfabc455485687a94affd8fa3a45 P 097ced8c992f4403968cf854115f981a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b5d85efc1b14ff0a9f67ea5af63ad1c (127.24.76.67:34895), bbad78bf66044c14beead1caf3caf528 (127.24.76.66:46529)
I20260808 11:42:27.590227 20947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:27.584119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:43655 (local address 127.24.76.66:46529)
0808 11:42:27.584320 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:27.584327 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:27.584968 (+ 641us) server_negotiation.cc:407] Connection header received
0808 11:42:27.585162 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:27.585167 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:27.585239 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:27.585316 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:42:27.585887 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:27.586616 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:27.588607 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:27.589332 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:27.589482 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:27.589555 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:27.589820 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 11:42:27.589960 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 11:42:27.590242 20945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:27.584046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:59795 (local address 127.24.76.67:34895)
0808 11:42:27.584252 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:27.584259 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:27.585110 (+ 851us) server_negotiation.cc:407] Connection header received
0808 11:42:27.585263 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:27.585268 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:27.585328 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:27.585402 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:42:27.586100 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:27.586805 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:27.588586 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:27.589345 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:27.589533 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:27.589604 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:27.589867 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 11:42:27.590021 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 11:42:27.590576 20960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:27.583926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:46529 (local address 127.24.76.65:43655)
0808 11:42:27.584849 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:27.584873 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:27.585021 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:27.585351 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:27.585356 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:27.585381 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:42:27.585742 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:27.585755 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:27.586749 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:27.586755 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:27.588387 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:27.588400 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:27.589528 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:27.589534 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:27.589671 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:27.589694 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 11:42:27.589773 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 11:42:27.589935 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":811,"mutex_wait_us":120,"thread_start_us":600,"threads_started":1}
I20260808 11:42:27.590615 20962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:27.584020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:34895 (local address 127.24.76.65:59795)
0808 11:42:27.584999 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:27.585013 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:27.585137 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:27.585428 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:27.585432 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:27.585506 (+ 74us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:42:27.585929 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:27.585938 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:27.586940 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:27.586945 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:27.588383 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:27.588395 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:27.589578 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:27.589586 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:27.589721 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:27.589743 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 11:42:27.589820 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 11:42:27.589963 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":879,"mutex_wait_us":23,"spinlock_wait_cycles":2304,"thread_start_us":565,"threads_started":1}
I20260808 11:42:27.591655 20852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dacdfabc455485687a94affd8fa3a45" candidate_uuid: "097ced8c992f4403968cf854115f981a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b5d85efc1b14ff0a9f67ea5af63ad1c" is_pre_election: true
I20260808 11:42:27.591655 20636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dacdfabc455485687a94affd8fa3a45" candidate_uuid: "097ced8c992f4403968cf854115f981a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbad78bf66044c14beead1caf3caf528" is_pre_election: true
I20260808 11:42:27.592254 20636 raft_consensus.cc:2454] T 2dacdfabc455485687a94affd8fa3a45 P bbad78bf66044c14beead1caf3caf528 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 097ced8c992f4403968cf854115f981a in term 0.
I20260808 11:42:27.592253 20852 raft_consensus.cc:2454] T 2dacdfabc455485687a94affd8fa3a45 P 9b5d85efc1b14ff0a9f67ea5af63ad1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 097ced8c992f4403968cf854115f981a in term 0.
I20260808 11:42:27.592902 20315 leader_election.cc:302] T 2dacdfabc455485687a94affd8fa3a45 P 097ced8c992f4403968cf854115f981a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 097ced8c992f4403968cf854115f981a, bbad78bf66044c14beead1caf3caf528; no voters:
I20260808 11:42:27.593257 20954 raft_consensus.cc:2781] T 2dacdfabc455485687a94affd8fa3a45 P 097ced8c992f4403968cf854115f981a [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260808 11:42:27.593320 20714 tablet.cc:2369] T 2dacdfabc455485687a94affd8fa3a45 P bbad78bf66044c14beead1caf3caf528: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:42:27.593436 20954 raft_consensus.cc:482] T 2dacdfabc455485687a94affd8fa3a45 P 097ced8c992f4403968cf854115f981a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:42:27.593555 20954 raft_consensus.cc:3037] T 2dacdfabc455485687a94affd8fa3a45 P 097ced8c992f4403968cf854115f981a [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:42:27.596024 20954 raft_consensus.cc:504] T 2dacdfabc455485687a94affd8fa3a45 P 097ced8c992f4403968cf854115f981a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "097ced8c992f4403968cf854115f981a" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 37931 } } peers { permanent_uuid: "9b5d85efc1b14ff0a9f67ea5af63ad1c" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 34895 } } peers { permanent_uuid: "bbad78bf66044c14beead1caf3caf528" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 46529 } }
I20260808 11:42:27.596588 20954 leader_election.cc:288] T 2dacdfabc455485687a94affd8fa3a45 P 097ced8c992f4403968cf854115f981a [CANDIDATE]: Term 1 election: Requested vote from peers 9b5d85efc1b14ff0a9f67ea5af63ad1c (127.24.76.67:34895), bbad78bf66044c14beead1caf3caf528 (127.24.76.66:46529)
I20260808 11:42:27.596947 20852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dacdfabc455485687a94affd8fa3a45" candidate_uuid: "097ced8c992f4403968cf854115f981a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b5d85efc1b14ff0a9f67ea5af63ad1c"
I20260808 11:42:27.597018 20636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dacdfabc455485687a94affd8fa3a45" candidate_uuid: "097ced8c992f4403968cf854115f981a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbad78bf66044c14beead1caf3caf528"
I20260808 11:42:27.597121 20852 raft_consensus.cc:3037] T 2dacdfabc455485687a94affd8fa3a45 P 9b5d85efc1b14ff0a9f67ea5af63ad1c [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:42:27.597211 20636 raft_consensus.cc:3037] T 2dacdfabc455485687a94affd8fa3a45 P bbad78bf66044c14beead1caf3caf528 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:42:27.599481 20636 raft_consensus.cc:2454] T 2dacdfabc455485687a94affd8fa3a45 P bbad78bf66044c14beead1caf3caf528 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 097ced8c992f4403968cf854115f981a in term 1.
I20260808 11:42:27.599525 20852 raft_consensus.cc:2454] T 2dacdfabc455485687a94affd8fa3a45 P 9b5d85efc1b14ff0a9f67ea5af63ad1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 097ced8c992f4403968cf854115f981a in term 1.
I20260808 11:42:27.599826 20315 leader_election.cc:302] T 2dacdfabc455485687a94affd8fa3a45 P 097ced8c992f4403968cf854115f981a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 097ced8c992f4403968cf854115f981a, bbad78bf66044c14beead1caf3caf528; no voters:
I20260808 11:42:27.600106 20954 raft_consensus.cc:2781] T 2dacdfabc455485687a94affd8fa3a45 P 097ced8c992f4403968cf854115f981a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:42:27.600517 20954 raft_consensus.cc:686] T 2dacdfabc455485687a94affd8fa3a45 P 097ced8c992f4403968cf854115f981a [term 1 LEADER]: Becoming Leader. State: Replica: 097ced8c992f4403968cf854115f981a, State: Running, Role: LEADER
I20260808 11:42:27.600936 20954 consensus_queue.cc:237] T 2dacdfabc455485687a94affd8fa3a45 P 097ced8c992f4403968cf854115f981a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "097ced8c992f4403968cf854115f981a" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 37931 } } peers { permanent_uuid: "9b5d85efc1b14ff0a9f67ea5af63ad1c" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 34895 } } peers { permanent_uuid: "bbad78bf66044c14beead1caf3caf528" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 46529 } }
I20260808 11:42:27.605515 20187 catalog_manager.cc:5495] T 2dacdfabc455485687a94affd8fa3a45 P 097ced8c992f4403968cf854115f981a reported cstate change: term changed from 0 to 1, leader changed from <none> to 097ced8c992f4403968cf854115f981a (127.24.76.65). New cstate: current_term: 1 leader_uuid: "097ced8c992f4403968cf854115f981a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "097ced8c992f4403968cf854115f981a" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 37931 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9b5d85efc1b14ff0a9f67ea5af63ad1c" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 34895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbad78bf66044c14beead1caf3caf528" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 46529 } health_report { overall_health: UNKNOWN } } }
W20260808 11:42:27.625681 20495 tablet.cc:2369] T 2dacdfabc455485687a94affd8fa3a45 P 097ced8c992f4403968cf854115f981a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:42:27.652539 20943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:27.647454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43212 (local address 127.24.76.65:37931)
0808 11:42:27.647688 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:27.647693 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:27.647711 (+ 18us) server_negotiation.cc:407] Connection header received
0808 11:42:27.647883 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:27.647886 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:27.647981 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:27.648102 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:42:27.648379 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:27.649058 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:27.650481 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:27.650886 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:27.650982 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:27.651455 (+ 473us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:42:27.652008 (+ 553us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:42:27.652128 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:27.652220 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 11:42:27.652285 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 11:42:27.660537 20852 raft_consensus.cc:1261] T 2dacdfabc455485687a94affd8fa3a45 P 9b5d85efc1b14ff0a9f67ea5af63ad1c [term 1 FOLLOWER]: Refusing update from remote peer 097ced8c992f4403968cf854115f981a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:42:27.660537 20636 raft_consensus.cc:1261] T 2dacdfabc455485687a94affd8fa3a45 P bbad78bf66044c14beead1caf3caf528 [term 1 FOLLOWER]: Refusing update from remote peer 097ced8c992f4403968cf854115f981a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:42:27.661445 20964 consensus_queue.cc:1035] T 2dacdfabc455485687a94affd8fa3a45 P 097ced8c992f4403968cf854115f981a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b5d85efc1b14ff0a9f67ea5af63ad1c" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 34895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:42:27.661780 20954 consensus_queue.cc:1035] T 2dacdfabc455485687a94affd8fa3a45 P 097ced8c992f4403968cf854115f981a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbad78bf66044c14beead1caf3caf528" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:42:27.672703 20970 mvcc.cc:205] Tried to move back new op lower bound from 7316231568008151040 to 7316231567776858112. Current Snapshot: MvccSnapshot[applied={T|T < 7316231568008151040}]
I20260808 11:42:27.686136 20188 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:42:27.686404 20188 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:42:27.689838 20188 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 67ef9188482d4a94a509f4e30df7e7b1: Sending DeleteTablet for 3 replicas of tablet 2dacdfabc455485687a94affd8fa3a45
I20260808 11:42:27.691073 20386 tablet_service.cc:1514] Processing DeleteTablet for tablet 2dacdfabc455485687a94affd8fa3a45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:42:27 UTC) from {username='slave'} at 127.0.0.1:43206
I20260808 11:42:27.691200 20822 tablet_service.cc:1514] Processing DeleteTablet for tablet 2dacdfabc455485687a94affd8fa3a45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:42:27 UTC) from {username='slave'} at 127.0.0.1:43386
I20260808 11:42:27.691399 20602 tablet_service.cc:1514] Processing DeleteTablet for tablet 2dacdfabc455485687a94affd8fa3a45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:42:27 UTC) from {username='slave'} at 127.0.0.1:40658
I20260808 11:42:27.694332 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 20290
I20260808 11:42:27.707300 20991 tablet_replica.cc:330] T 2dacdfabc455485687a94affd8fa3a45 P bbad78bf66044c14beead1caf3caf528: stopping tablet replica
I20260808 11:42:27.707821 20991 raft_consensus.cc:2229] T 2dacdfabc455485687a94affd8fa3a45 P bbad78bf66044c14beead1caf3caf528 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:42:27.708379 20991 raft_consensus.cc:2258] T 2dacdfabc455485687a94affd8fa3a45 P bbad78bf66044c14beead1caf3caf528 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:42:27.709864 20987 tablet_replica.cc:330] T 2dacdfabc455485687a94affd8fa3a45 P 097ced8c992f4403968cf854115f981a: stopping tablet replica
I20260808 11:42:27.710243 20989 tablet_replica.cc:330] T 2dacdfabc455485687a94affd8fa3a45 P 9b5d85efc1b14ff0a9f67ea5af63ad1c: stopping tablet replica
I20260808 11:42:27.710328 20987 raft_consensus.cc:2229] T 2dacdfabc455485687a94affd8fa3a45 P 097ced8c992f4403968cf854115f981a [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:42:27.710779 20989 raft_consensus.cc:2229] T 2dacdfabc455485687a94affd8fa3a45 P 9b5d85efc1b14ff0a9f67ea5af63ad1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:42:27.711050 20987 raft_consensus.cc:2258] T 2dacdfabc455485687a94affd8fa3a45 P 097ced8c992f4403968cf854115f981a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:42:27.711396 20989 raft_consensus.cc:2258] T 2dacdfabc455485687a94affd8fa3a45 P 9b5d85efc1b14ff0a9f67ea5af63ad1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:42:27.711719 20991 ts_tablet_manager.cc:1905] T 2dacdfabc455485687a94affd8fa3a45 P bbad78bf66044c14beead1caf3caf528: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:42:27.713850 20989 ts_tablet_manager.cc:1905] T 2dacdfabc455485687a94affd8fa3a45 P 9b5d85efc1b14ff0a9f67ea5af63ad1c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:42:27.717329 20987 ts_tablet_manager.cc:1905] T 2dacdfabc455485687a94affd8fa3a45 P 097ced8c992f4403968cf854115f981a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:42:27.718812 20991 ts_tablet_manager.cc:1918] T 2dacdfabc455485687a94affd8fa3a45 P bbad78bf66044c14beead1caf3caf528: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:42:27.719007 20991 log.cc:1196] T 2dacdfabc455485687a94affd8fa3a45 P bbad78bf66044c14beead1caf3caf528: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-1/wal/wals/2dacdfabc455485687a94affd8fa3a45
I20260808 11:42:27.719597 20991 ts_tablet_manager.cc:1939] T 2dacdfabc455485687a94affd8fa3a45 P bbad78bf66044c14beead1caf3caf528: Deleting consensus metadata
I20260808 11:42:27.721197 20989 ts_tablet_manager.cc:1918] T 2dacdfabc455485687a94affd8fa3a45 P 9b5d85efc1b14ff0a9f67ea5af63ad1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:42:27.721426 20989 log.cc:1196] T 2dacdfabc455485687a94affd8fa3a45 P 9b5d85efc1b14ff0a9f67ea5af63ad1c: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-2/wal/wals/2dacdfabc455485687a94affd8fa3a45
I20260808 11:42:27.722028 20989 ts_tablet_manager.cc:1939] T 2dacdfabc455485687a94affd8fa3a45 P 9b5d85efc1b14ff0a9f67ea5af63ad1c: Deleting consensus metadata
I20260808 11:42:27.722028 20165 catalog_manager.cc:4841] TS bbad78bf66044c14beead1caf3caf528 (127.24.76.66:46529): tablet 2dacdfabc455485687a94affd8fa3a45 (table test-table [id=75bcdcc9e2654119820c21a856e661dc]) successfully deleted
I20260808 11:42:27.723953 20161 catalog_manager.cc:4841] TS 9b5d85efc1b14ff0a9f67ea5af63ad1c (127.24.76.67:34895): tablet 2dacdfabc455485687a94affd8fa3a45 (table test-table [id=75bcdcc9e2654119820c21a856e661dc]) successfully deleted
I20260808 11:42:27.725581 20987 ts_tablet_manager.cc:1918] T 2dacdfabc455485687a94affd8fa3a45 P 097ced8c992f4403968cf854115f981a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:42:27.726089 20943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:27.696272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43224 (local address 127.24.76.65:37931)
0808 11:42:27.696467 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:27.696474 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:27.709187 (+ 12713us) server_negotiation.cc:407] Connection header received
0808 11:42:27.709773 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:27.709778 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:27.709914 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:27.710048 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:27.711975 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:27.713225 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:27.718025 (+ 4800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:27.718734 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:27.718848 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:27.724182 (+ 5334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:27.724194 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:27.724212 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:42:27.725088 (+ 876us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:27.725149 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:42:27.725177 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:27.725278 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:27.725573 (+ 295us) server_negotiation.cc:299] Negotiation successful
0808 11:42:27.725686 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 11:42:27.725752 20987 log.cc:1196] T 2dacdfabc455485687a94affd8fa3a45 P 097ced8c992f4403968cf854115f981a: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786189042529715-24881-0/minicluster-data/ts-0/wal/wals/2dacdfabc455485687a94affd8fa3a45
I20260808 11:42:27.726742 20987 ts_tablet_manager.cc:1939] T 2dacdfabc455485687a94affd8fa3a45 P 097ced8c992f4403968cf854115f981a: Deleting consensus metadata
I20260808 11:42:27.726814 20484 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:42:27.824837 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 20290
I20260808 11:42:27.825114 20162 catalog_manager.cc:4841] TS 097ced8c992f4403968cf854115f981a (127.24.76.65:37931): tablet 2dacdfabc455485687a94affd8fa3a45 (table test-table [id=75bcdcc9e2654119820c21a856e661dc]) successfully deleted
I20260808 11:42:27.857596 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 20498
I20260808 11:42:27.865284 20947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:27.858500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40670 (local address 127.24.76.66:46529)
0808 11:42:27.858901 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:27.858907 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:27.858928 (+ 21us) server_negotiation.cc:407] Connection header received
0808 11:42:27.859025 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:27.859031 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:27.859123 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:27.859220 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:27.859581 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:27.860771 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:27.862087 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:27.862737 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:27.862856 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:27.863817 (+ 961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:27.863830 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:27.863847 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:42:27.864562 (+ 715us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:27.864623 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:42:27.864647 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:27.864764 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:27.864811 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 11:42:27.864911 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276}
I20260808 11:42:27.865940 20707 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:42:27.952418 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 20498
I20260808 11:42:27.979121 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 20718
I20260808 11:42:27.985592 20945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:27.979856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43396 (local address 127.24.76.67:34895)
0808 11:42:27.980023 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:27.980028 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:27.980095 (+ 67us) server_negotiation.cc:407] Connection header received
0808 11:42:27.980259 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:27.980263 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:27.980363 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:27.980448 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:27.980814 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:27.981602 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:27.982921 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:27.983306 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:27.983401 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:27.984385 (+ 984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:27.984396 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:27.984413 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:42:27.984995 (+ 582us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:27.985045 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:42:27.985066 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:27.985152 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:27.985239 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 11:42:27.985334 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 11:42:27.986254 20917 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:42:28.073654 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 20718
I20260808 11:42:28.094492 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 20134
I20260808 11:42:28.095600 20235 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:42:28.207763 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 20134
2026-08-08T11:42:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2910 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:42:28 dist-test-slave-tc2s krb5kdc[21002](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:42:28 dist-test-slave-tc2s krb5kdc[21002](info): set up 2 sockets
Aug 08 11:42:28 dist-test-slave-tc2s krb5kdc[21002](info): commencing operation
krb5kdc: starting...
W20260808 11:42:29.439260 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.005s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:42:29 dist-test-slave-tc2s krb5kdc[21002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:42:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:42:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
I20260808 11:42:29.718546 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:42265
--webserver_interface=127.24.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:42303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:42265
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:42:30.052659 21029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:42:30.052958 21029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:42:30.053089 21029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:42:30.059994 21029 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:42:30.060112 21029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:42:30.060189 21029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:42:30.060266 21029 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:42:30.060348 21029 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:42:30.070658 21029 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:42303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:42265
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:42265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:42:30.071918 21029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:42:30.073176 21029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:42:30.084159 21035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:30.084242 21038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:30.084228 21036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:42:30.085628 21029 server_base.cc:1034] running on GCE node
I20260808 11:42:30.086395 21029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:42:30.087774 21029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:42:30.088975 21029 hybrid_clock.cc:648] HybridClock initialized: now 1786189350088927 us; error 39 us; skew 500 ppm
I20260808 11:42:30.089298 21029 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:42:30 dist-test-slave-tc2s krb5kdc[21002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189350, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:42:30.092346 21029 init.cc:377] Logged in from keytab as kudu/127.24.76.126@KRBTEST.COM (short username kudu)
I20260808 11:42:30.093693 21029 webserver.cc:460] Webserver started at http://127.24.76.126:45371/ using document root <none> and password file <none>
I20260808 11:42:30.094374 21029 fs_manager.cc:359] Metadata directory not provided
I20260808 11:42:30.094472 21029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:42:30.094715 21029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:42:30.097582 21029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "1b187010b22f4f94a964526ed0d1f71e"
format_stamp: "Formatted at 2026-08-08 11:42:30 on dist-test-slave-tc2s"
server_key: "4c72e64bd9df11013fb9043d769d2f1e"
server_key_iv: "0aacf95888a5c815a7bb3faf63bcb35d"
server_key_version: "encryptionkey@0"
I20260808 11:42:30.098318 21029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "1b187010b22f4f94a964526ed0d1f71e"
format_stamp: "Formatted at 2026-08-08 11:42:30 on dist-test-slave-tc2s"
server_key: "4c72e64bd9df11013fb9043d769d2f1e"
server_key_iv: "0aacf95888a5c815a7bb3faf63bcb35d"
server_key_version: "encryptionkey@0"
I20260808 11:42:30.103193 21029 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 11:42:30.106391 21052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:30.107575 21029 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260808 11:42:30.107723 21029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "1b187010b22f4f94a964526ed0d1f71e"
format_stamp: "Formatted at 2026-08-08 11:42:30 on dist-test-slave-tc2s"
server_key: "4c72e64bd9df11013fb9043d769d2f1e"
server_key_iv: "0aacf95888a5c815a7bb3faf63bcb35d"
server_key_version: "encryptionkey@0"
I20260808 11:42:30.107889 21029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:42:30.135435 21029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:42:30.139135 21029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:42:30.139394 21029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:42:30.155733 21029 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:42265
I20260808 11:42:30.155763 21142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:42265 every 8 connection(s)
I20260808 11:42:30.156939 21029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:42:30.160454 21144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:42:30.164386 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 21029
I20260808 11:42:30.164554 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:42:30.164984 24881 external_mini_cluster.cc:1428] Setting key 6658cc61f3f53b2b15932e175cb70534
I20260808 11:42:30.171062 21144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b187010b22f4f94a964526ed0d1f71e: Bootstrap starting.
I20260808 11:42:30.174599 21144 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b187010b22f4f94a964526ed0d1f71e: Neither blocks nor log segments found. Creating new log.
I20260808 11:42:30.175751 21144 log.cc:824] T 00000000000000000000000000000000 P 1b187010b22f4f94a964526ed0d1f71e: Log is configured to *not* fsync() on all Append() calls
I20260808 11:42:30.179055 21144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b187010b22f4f94a964526ed0d1f71e: No bootstrap required, opened a new log
Aug 08 11:42:30 dist-test-slave-tc2s krb5kdc[21002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:42:30.186590 21144 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b187010b22f4f94a964526ed0d1f71e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b187010b22f4f94a964526ed0d1f71e" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 42265 } }
I20260808 11:42:30.186931 21144 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b187010b22f4f94a964526ed0d1f71e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:42:30.187095 21144 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b187010b22f4f94a964526ed0d1f71e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b187010b22f4f94a964526ed0d1f71e, State: Initialized, Role: FOLLOWER
I20260808 11:42:30.187614 21144 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b187010b22f4f94a964526ed0d1f71e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b187010b22f4f94a964526ed0d1f71e" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 42265 } }
I20260808 11:42:30.187863 21144 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b187010b22f4f94a964526ed0d1f71e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:42:30.188014 21144 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b187010b22f4f94a964526ed0d1f71e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:42:30.188192 21144 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b187010b22f4f94a964526ed0d1f71e [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:42:30.188215 21149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:30.166672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34408 (local address 127.24.76.126:42265)
0808 11:42:30.167441 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:30.167451 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:30.167491 (+ 40us) server_negotiation.cc:407] Connection header received
0808 11:42:30.169669 (+ 2178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:30.169691 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:30.170097 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:30.170393 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:30.172276 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:30.173576 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:30.175393 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:30.176107 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:30.176307 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:30.181053 (+ 4746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:30.181061 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:30.181074 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:42:30.181112 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:30.184131 (+ 3019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:30.184799 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:30.184806 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:30.184812 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:30.184992 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:30.185368 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:30.185371 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:30.185373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:30.185956 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:30.186237 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:30.186693 (+ 456us) server_negotiation.cc:299] Negotiation successful
0808 11:42:30.187103 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":352,"threads_started":1}
I20260808 11:42:30.190485 21144 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b187010b22f4f94a964526ed0d1f71e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b187010b22f4f94a964526ed0d1f71e" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 42265 } }
I20260808 11:42:30.190930 21144 leader_election.cc:302] T 00000000000000000000000000000000 P 1b187010b22f4f94a964526ed0d1f71e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b187010b22f4f94a964526ed0d1f71e; no voters:
I20260808 11:42:30.191429 21144 leader_election.cc:288] T 00000000000000000000000000000000 P 1b187010b22f4f94a964526ed0d1f71e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:42:30.191541 21152 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b187010b22f4f94a964526ed0d1f71e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:42:30.192139 21152 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b187010b22f4f94a964526ed0d1f71e [term 1 LEADER]: Becoming Leader. State: Replica: 1b187010b22f4f94a964526ed0d1f71e, State: Running, Role: LEADER
I20260808 11:42:30.192624 21152 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b187010b22f4f94a964526ed0d1f71e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b187010b22f4f94a964526ed0d1f71e" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 42265 } }
I20260808 11:42:30.193248 21144 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b187010b22f4f94a964526ed0d1f71e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:42:30.195001 21156 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b187010b22f4f94a964526ed0d1f71e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b187010b22f4f94a964526ed0d1f71e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b187010b22f4f94a964526ed0d1f71e" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 42265 } } }
I20260808 11:42:30.195317 21156 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b187010b22f4f94a964526ed0d1f71e [sys.catalog]: This master's current role is: LEADER
I20260808 11:42:30.196681 21164 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:42:30.196754 21160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b187010b22f4f94a964526ed0d1f71e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b187010b22f4f94a964526ed0d1f71e. Latest consensus state: current_term: 1 leader_uuid: "1b187010b22f4f94a964526ed0d1f71e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b187010b22f4f94a964526ed0d1f71e" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 42265 } } }
I20260808 11:42:30.197013 21160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b187010b22f4f94a964526ed0d1f71e [sys.catalog]: This master's current role is: LEADER
I20260808 11:42:30.201650 21164 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:42:30.210362 21164 catalog_manager.cc:1347] Generated new cluster ID: 82500d858f234a75a7b5b286796425d3
I20260808 11:42:30.210466 21164 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:42:30.229673 21164 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:42:30.231271 21164 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:42:30.241153 21164 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b187010b22f4f94a964526ed0d1f71e: Generated new TSK 0
I20260808 11:42:30.241976 21164 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab.
Entry for principal kudu/127.24.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab.
I20260808 11:42:30.323184 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.65:0
--local_ip_for_outbound_sockets=127.24.76.65
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:42265
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:42303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:42:30.696045 21189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:42:30.696295 21189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:42:30.696420 21189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:42:30.703572 21189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:42:30.703668 21189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:42:30.703822 21189 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.65
I20260808 11:42:30.713215 21189 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:42303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:42265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.65
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:42:30.714442 21189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:42:30.715799 21189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:42:30.728065 21200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:30.728097 21194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:30.728030 21195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:42:30.728801 21189 server_base.cc:1034] running on GCE node
I20260808 11:42:30.729321 21189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:42:30.730192 21189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:42:30.731381 21189 hybrid_clock.cc:648] HybridClock initialized: now 1786189350731339 us; error 34 us; skew 500 ppm
I20260808 11:42:30.731669 21189 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:42:30 dist-test-slave-tc2s krb5kdc[21002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189350, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:42:30.734656 21189 init.cc:377] Logged in from keytab as kudu/127.24.76.65@KRBTEST.COM (short username kudu)
I20260808 11:42:30.735994 21189 webserver.cc:460] Webserver started at http://127.24.76.65:39137/ using document root <none> and password file <none>
I20260808 11:42:30.736618 21189 fs_manager.cc:359] Metadata directory not provided
I20260808 11:42:30.736732 21189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:42:30.737030 21189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:42:30.739840 21189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-0/data/instance:
uuid: "024c9160a8a945a5bcec70c056230538"
format_stamp: "Formatted at 2026-08-08 11:42:30 on dist-test-slave-tc2s"
server_key: "e8c24862c65ae987d633788ccbb786fb"
server_key_iv: "a188896c358a1392cd6df8f22c54a18b"
server_key_version: "encryptionkey@0"
I20260808 11:42:30.740659 21189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance:
uuid: "024c9160a8a945a5bcec70c056230538"
format_stamp: "Formatted at 2026-08-08 11:42:30 on dist-test-slave-tc2s"
server_key: "e8c24862c65ae987d633788ccbb786fb"
server_key_iv: "a188896c358a1392cd6df8f22c54a18b"
server_key_version: "encryptionkey@0"
I20260808 11:42:30.745954 21189 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 11:42:30.749414 21208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:30.750823 21189 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 11:42:30.750993 21189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-0/wal
uuid: "024c9160a8a945a5bcec70c056230538"
format_stamp: "Formatted at 2026-08-08 11:42:30 on dist-test-slave-tc2s"
server_key: "e8c24862c65ae987d633788ccbb786fb"
server_key_iv: "a188896c358a1392cd6df8f22c54a18b"
server_key_version: "encryptionkey@0"
I20260808 11:42:30.751190 21189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:42:30.765367 21189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:42:30.769016 21189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:42:30.769273 21189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:42:30.770200 21189 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:42:30.771494 21189 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:42:30.771581 21189 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:30.771701 21189 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:42:30.771760 21189 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:30.801687 21189 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.65:44363
I20260808 11:42:30.801726 21394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.65:44363 every 8 connection(s)
I20260808 11:42:30.802963 21189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
I20260808 11:42:30.806691 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 21189
I20260808 11:42:30.806893 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance
I20260808 11:42:30.807443 24881 external_mini_cluster.cc:1428] Setting key c2e86248ec70c3adfc1952a6e19dacd1
Aug 08 11:42:30 dist-test-slave-tc2s krb5kdc[21002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189350, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:42:30.823348 21399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:30.805426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:40299 (local address 127.24.76.126:42265)
0808 11:42:30.805858 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:30.805862 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:30.806843 (+ 981us) server_negotiation.cc:407] Connection header received
0808 11:42:30.809050 (+ 2207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:30.809055 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:30.809121 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:30.809231 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:30.810852 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:30.811946 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:30.813546 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:30.814104 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:30.814229 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:30.817583 (+ 3354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:30.817590 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:30.817594 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:42:30.817632 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:30.820488 (+ 2856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:30.821147 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:30.821152 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:30.821155 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:30.821306 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:30.821684 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:30.821688 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:30.821691 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:30.822059 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:30.822193 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:30.822819 (+ 626us) server_negotiation.cc:299] Negotiation successful
0808 11:42:30.823022 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":209,"threads_started":1}
I20260808 11:42:30.824173 21400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:30.805985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:42265 (local address 127.24.76.65:40299)
0808 11:42:30.806695 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:30.806741 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:30.808746 (+ 2005us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:30.809514 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:30.809525 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:30.810117 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:42:30.810674 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:30.810691 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:30.812090 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:30.812094 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:30.813405 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:30.813418 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:30.814293 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:30.814298 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:30.814491 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:30.815112 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:42:30.815134 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:42:30.817369 (+ 2235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:42:30.820620 (+ 3251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:42:30.820627 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:42:30.820639 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:42:30.821038 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:42:30.821399 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:42:30.821402 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:42:30.821404 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:42:30.821589 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:42:30.822161 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:42:30.822167 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:42:30.822466 (+ 299us) client_negotiation.cc:769] Sending connection context
0808 11:42:30.822696 (+ 230us) client_negotiation.cc:240] Negotiation successful
0808 11:42:30.823074 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":313,"threads_started":1}
I20260808 11:42:30.825651 21395 heartbeater.cc:347] Connected to a master server at 127.24.76.126:42265
I20260808 11:42:30.825973 21395 heartbeater.cc:464] Registering TS with master...
I20260808 11:42:30.826668 21395 heartbeater.cc:511] Master 127.24.76.126:42265 requested a full tablet report, sending...
I20260808 11:42:30.828994 21080 ts_manager.cc:194] Registered new tserver with Master: 024c9160a8a945a5bcec70c056230538 (127.24.76.65:44363)
I20260808 11:42:30.830911 21080 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.65@KRBTEST.COM'} at 127.24.76.65:40299
WARNING: no policy specified for kudu/127.24.76.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab.
Entry for principal kudu/127.24.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab.
I20260808 11:42:30.877101 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.66:0
--local_ip_for_outbound_sockets=127.24.76.66
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:42265
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:42303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:42:31.208119 21406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:42:31.208421 21406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:42:31.208560 21406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:42:31.215674 21406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:42:31.215771 21406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:42:31.215919 21406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.66
I20260808 11:42:31.225171 21406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:42303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:42265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.66
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:42:31.226377 21406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:42:31.227607 21406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:42:31.240015 21413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:31.239959 21414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:31.239989 21416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:42:31.240775 21406 server_base.cc:1034] running on GCE node
I20260808 11:42:31.241421 21406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:42:31.242350 21406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:42:31.243538 21406 hybrid_clock.cc:648] HybridClock initialized: now 1786189351243496 us; error 31 us; skew 500 ppm
I20260808 11:42:31.243811 21406 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:42:31 dist-test-slave-tc2s krb5kdc[21002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189351, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:42:31.246701 21406 init.cc:377] Logged in from keytab as kudu/127.24.76.66@KRBTEST.COM (short username kudu)
I20260808 11:42:31.247998 21406 webserver.cc:460] Webserver started at http://127.24.76.66:44861/ using document root <none> and password file <none>
I20260808 11:42:31.248595 21406 fs_manager.cc:359] Metadata directory not provided
I20260808 11:42:31.248708 21406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:42:31.249001 21406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:42:31.251754 21406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-1/data/instance:
uuid: "0525df487545469fb2d53500c2d77270"
format_stamp: "Formatted at 2026-08-08 11:42:31 on dist-test-slave-tc2s"
server_key: "29f9ba7197b3c1ddc5e1ad0ac08af6ca"
server_key_iv: "209a98ba7fcff7b01110524826a13cab"
server_key_version: "encryptionkey@0"
I20260808 11:42:31.252528 21406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance:
uuid: "0525df487545469fb2d53500c2d77270"
format_stamp: "Formatted at 2026-08-08 11:42:31 on dist-test-slave-tc2s"
server_key: "29f9ba7197b3c1ddc5e1ad0ac08af6ca"
server_key_iv: "209a98ba7fcff7b01110524826a13cab"
server_key_version: "encryptionkey@0"
I20260808 11:42:31.257611 21406 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 11:42:31.260871 21428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:31.262099 21406 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:42:31.262245 21406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-1/wal
uuid: "0525df487545469fb2d53500c2d77270"
format_stamp: "Formatted at 2026-08-08 11:42:31 on dist-test-slave-tc2s"
server_key: "29f9ba7197b3c1ddc5e1ad0ac08af6ca"
server_key_iv: "209a98ba7fcff7b01110524826a13cab"
server_key_version: "encryptionkey@0"
I20260808 11:42:31.262426 21406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:42:31.288468 21406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:42:31.292392 21406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:42:31.292654 21406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:42:31.293591 21406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:42:31.294945 21406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:42:31.295025 21406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:31.295145 21406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:42:31.295197 21406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:31.325057 21406 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.66:39985
I20260808 11:42:31.325110 21617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.66:39985 every 8 connection(s)
I20260808 11:42:31.326334 21406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
I20260808 11:42:31.334719 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 21406
I20260808 11:42:31.334909 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance
I20260808 11:42:31.335491 24881 external_mini_cluster.cc:1428] Setting key 03d3905bbd99ebf7efcb8720eaa0dce0
Aug 08 11:42:31 dist-test-slave-tc2s krb5kdc[21002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189351, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:42:31.345944 21620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:31.328659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:37913 (local address 127.24.76.126:42265)
0808 11:42:31.329063 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:31.329067 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:31.329928 (+ 861us) server_negotiation.cc:407] Connection header received
0808 11:42:31.331736 (+ 1808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:31.331741 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:31.331819 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:31.331915 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:31.333371 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:31.334137 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:31.336294 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:31.336857 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:31.336978 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:31.340477 (+ 3499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:31.340483 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:31.340486 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:42:31.340518 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:31.343061 (+ 2543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:31.343732 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:31.343736 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:31.343739 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:31.343866 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:31.344283 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:31.344287 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:31.344289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:31.344546 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:31.344677 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:31.345318 (+ 641us) server_negotiation.cc:299] Negotiation successful
0808 11:42:31.345533 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":298,"thread_start_us":204,"threads_started":1}
I20260808 11:42:31.346863 21622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:31.329105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:42265 (local address 127.24.76.66:37913)
0808 11:42:31.329755 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:31.329795 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:31.331469 (+ 1674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:31.332158 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:31.332166 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:31.332660 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:42:31.333199 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:31.333217 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:31.334291 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:31.334295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:31.336150 (+ 1855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:31.336168 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:31.337058 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:31.337063 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:31.337268 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:31.337954 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:42:31.337977 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:42:31.340240 (+ 2263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:42:31.343193 (+ 2953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:42:31.343202 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:42:31.343217 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:42:31.343618 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:42:31.343981 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:42:31.343986 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:42:31.343990 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:42:31.344173 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:42:31.344669 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:42:31.344676 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:42:31.344973 (+ 297us) client_negotiation.cc:769] Sending connection context
0808 11:42:31.345204 (+ 231us) client_negotiation.cc:240] Negotiation successful
0808 11:42:31.345689 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":244,"threads_started":1}
I20260808 11:42:31.348121 21618 heartbeater.cc:347] Connected to a master server at 127.24.76.126:42265
I20260808 11:42:31.348431 21618 heartbeater.cc:464] Registering TS with master...
I20260808 11:42:31.349207 21618 heartbeater.cc:511] Master 127.24.76.126:42265 requested a full tablet report, sending...
I20260808 11:42:31.350651 21080 ts_manager.cc:194] Registered new tserver with Master: 0525df487545469fb2d53500c2d77270 (127.24.76.66:39985)
I20260808 11:42:31.351600 21080 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.66@KRBTEST.COM'} at 127.24.76.66:37913
WARNING: no policy specified for kudu/127.24.76.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab.
Entry for principal kudu/127.24.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab.
I20260808 11:42:31.401558 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.67:0
--local_ip_for_outbound_sockets=127.24.76.67
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:42265
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:42303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:42:31.730361 21627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:42:31.730654 21627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:42:31.730788 21627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:42:31.737927 21627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:42:31.738055 21627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:42:31.738225 21627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.67
I20260808 11:42:31.747890 21627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:42303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:42265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.67
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:42:31.749202 21627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:42:31.750560 21627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:42:31.763195 21638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:31.763229 21633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:31.763257 21632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:42:31.764245 21627 server_base.cc:1034] running on GCE node
I20260808 11:42:31.764830 21627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:42:31.765777 21627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:42:31.766978 21627 hybrid_clock.cc:648] HybridClock initialized: now 1786189351766914 us; error 45 us; skew 500 ppm
I20260808 11:42:31.767262 21627 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:42:31 dist-test-slave-tc2s krb5kdc[21002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189351, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:42:31.770346 21627 init.cc:377] Logged in from keytab as kudu/127.24.76.67@KRBTEST.COM (short username kudu)
I20260808 11:42:31.771652 21627 webserver.cc:460] Webserver started at http://127.24.76.67:40625/ using document root <none> and password file <none>
I20260808 11:42:31.772300 21627 fs_manager.cc:359] Metadata directory not provided
I20260808 11:42:31.772416 21627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:42:31.772707 21627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:42:31.775645 21627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-2/data/instance:
uuid: "9d54c660bc2a44c0aeb2f4a2849ff955"
format_stamp: "Formatted at 2026-08-08 11:42:31 on dist-test-slave-tc2s"
server_key: "d57538c1f40afbaa7652179f1738206b"
server_key_iv: "d9a8a963c19abb4d2b88a324cb42ff54"
server_key_version: "encryptionkey@0"
I20260808 11:42:31.776455 21627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance:
uuid: "9d54c660bc2a44c0aeb2f4a2849ff955"
format_stamp: "Formatted at 2026-08-08 11:42:31 on dist-test-slave-tc2s"
server_key: "d57538c1f40afbaa7652179f1738206b"
server_key_iv: "d9a8a963c19abb4d2b88a324cb42ff54"
server_key_version: "encryptionkey@0"
I20260808 11:42:31.781384 21627 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 11:42:31.784678 21647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:31.785883 21627 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 11:42:31.786067 21627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-2/wal
uuid: "9d54c660bc2a44c0aeb2f4a2849ff955"
format_stamp: "Formatted at 2026-08-08 11:42:31 on dist-test-slave-tc2s"
server_key: "d57538c1f40afbaa7652179f1738206b"
server_key_iv: "d9a8a963c19abb4d2b88a324cb42ff54"
server_key_version: "encryptionkey@0"
I20260808 11:42:31.786247 21627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:42:31.798574 21627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:42:31.802835 21627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:42:31.803104 21627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:42:31.804040 21627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:42:31.805353 21627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:42:31.805430 21627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:31.805546 21627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:42:31.805604 21627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:31.833967 21395 heartbeater.cc:503] Master 127.24.76.126:42265 was elected leader, sending a full tablet report...
I20260808 11:42:31.836587 21627 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.67:44209
I20260808 11:42:31.836634 21847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.67:44209 every 8 connection(s)
I20260808 11:42:31.837859 21627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
I20260808 11:42:31.838922 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 21627
I20260808 11:42:31.839116 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance
I20260808 11:42:31.839674 24881 external_mini_cluster.cc:1428] Setting key ff5f12ebde20d1805c783db53d120a41
Aug 08 11:42:31 dist-test-slave-tc2s krb5kdc[21002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189351, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:42:31.856963 21620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:31.840603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:38215 (local address 127.24.76.126:42265)
0808 11:42:31.840808 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:31.840814 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:31.842264 (+ 1450us) server_negotiation.cc:407] Connection header received
0808 11:42:31.844144 (+ 1880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:31.844148 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:31.844213 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:31.844290 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:31.845693 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:31.846462 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:31.847917 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:31.848299 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:31.848391 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:31.851548 (+ 3157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:31.851553 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:31.851557 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:42:31.851587 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:31.854331 (+ 2744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:31.855119 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:31.855123 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:31.855125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:31.855243 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:31.855611 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:31.855616 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:31.855619 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:31.855876 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:31.855991 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:31.856568 (+ 577us) server_negotiation.cc:299] Negotiation successful
0808 11:42:31.856727 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 11:42:31.857935 21851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:31.841199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:42265 (local address 127.24.76.67:38215)
0808 11:42:31.842085 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:31.842149 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:31.843898 (+ 1749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:31.844475 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:31.844485 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:31.845008 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:42:31.845560 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:31.845578 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:31.846571 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:31.846574 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:31.847789 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:31.847801 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:31.848465 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:31.848469 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:31.848653 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:31.849246 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:42:31.849266 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:42:31.851361 (+ 2095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:42:31.854467 (+ 3106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:42:31.854475 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:42:31.854490 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:42:31.855021 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:42:31.855334 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:42:31.855337 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:42:31.855339 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:42:31.855521 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:42:31.855974 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:42:31.855985 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:42:31.856278 (+ 293us) client_negotiation.cc:769] Sending connection context
0808 11:42:31.856496 (+ 218us) client_negotiation.cc:240] Negotiation successful
0808 11:42:31.856859 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":333,"threads_started":1}
I20260808 11:42:31.859202 21848 heartbeater.cc:347] Connected to a master server at 127.24.76.126:42265
I20260808 11:42:31.859459 21848 heartbeater.cc:464] Registering TS with master...
I20260808 11:42:31.860110 21848 heartbeater.cc:511] Master 127.24.76.126:42265 requested a full tablet report, sending...
I20260808 11:42:31.861380 21080 ts_manager.cc:194] Registered new tserver with Master: 9d54c660bc2a44c0aeb2f4a2849ff955 (127.24.76.67:44209)
I20260808 11:42:31.862166 21080 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.67@KRBTEST.COM'} at 127.24.76.67:38215
I20260808 11:42:31.866819 24881 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:42:31.882310 21620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:31.870742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34422 (local address 127.24.76.126:42265)
0808 11:42:31.870916 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:31.870922 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:31.871346 (+ 424us) server_negotiation.cc:407] Connection header received
0808 11:42:31.871469 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:31.871472 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:31.871537 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:31.871613 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:31.872928 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:31.873608 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:31.874799 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:31.875189 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:31.875278 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:31.877691 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:31.877696 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:31.877699 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:42:31.877725 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:31.879945 (+ 2220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:31.880639 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:31.880642 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:31.880644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:31.880762 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:31.881085 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:31.881088 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:31.881090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:31.881345 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:31.881547 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:31.881819 (+ 272us) server_negotiation.cc:299] Negotiation successful
0808 11:42:31.881978 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 11:42:31.885741 21080 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34422:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:42:31.892575 21080 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:42:31.924979 21876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:31.904664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44714 (local address 127.24.76.65:44363)
0808 11:42:31.917085 (+ 12421us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:31.917093 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:31.917123 (+ 30us) server_negotiation.cc:407] Connection header received
0808 11:42:31.917217 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:31.917224 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:31.917489 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:31.917686 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:42:31.918437 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:31.920011 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:31.922597 (+ 2586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:31.923805 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:31.923941 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:31.924178 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:31.924486 (+ 308us) server_negotiation.cc:299] Negotiation successful
0808 11:42:31.924660 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12248,"thread_start_us":12088,"threads_started":1}
I20260808 11:42:31.925043 21873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:31.904165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:44363 (local address 127.0.0.1:44714)
0808 11:42:31.915913 (+ 11748us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:31.915931 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:31.916044 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:31.917723 (+ 1679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:31.917732 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:31.917767 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:42:31.918239 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:31.918253 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:31.920148 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:31.920154 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:31.922444 (+ 2290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:31.922458 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:31.924020 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:31.924026 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:31.924210 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:31.924272 (+ 62us) client_negotiation.cc:769] Sending connection context
0808 11:42:31.924402 (+ 130us) client_negotiation.cc:240] Negotiation successful
0808 11:42:31.924570 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11627,"mutex_wait_us":25,"thread_start_us":11488,"threads_started":1}
I20260808 11:42:31.925082 21874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:31.903945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:44209 (local address 127.0.0.1:37686)
0808 11:42:31.914641 (+ 10696us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:31.914693 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:31.914867 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:31.918911 (+ 4044us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:31.918915 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:31.918938 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:42:31.919248 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:31.919257 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:31.920789 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:31.920794 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:31.922980 (+ 2186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:31.922991 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:31.924304 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:31.924311 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:31.924475 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:31.924515 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 11:42:31.924571 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 11:42:31.924672 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10446,"mutex_wait_us":21,"thread_start_us":13455,"threads_started":1}
I20260808 11:42:31.926199 21872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:31.904096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37686 (local address 127.24.76.67:44209)
0808 11:42:31.918400 (+ 14304us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:31.918408 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:31.918437 (+ 29us) server_negotiation.cc:407] Connection header received
0808 11:42:31.918542 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:31.918549 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:31.918738 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:31.918896 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:42:31.919404 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:31.920649 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:31.923138 (+ 2489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:31.924087 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:31.925426 (+ 1339us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:31.925623 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:31.925696 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 11:42:31.925841 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14143,"thread_start_us":13981,"threads_started":1}
I20260808 11:42:31.926216 21871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:31.904397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:39985 (local address 127.0.0.1:55502)
0808 11:42:31.917680 (+ 13283us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:31.917696 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:31.917804 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:31.918450 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:31.918453 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:31.918479 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:42:31.918858 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:31.918871 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:31.920566 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:31.920570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:31.922914 (+ 2344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:31.922930 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:31.925430 (+ 2500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:31.925437 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:31.925603 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:31.925629 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 11:42:31.925707 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 11:42:31.925825 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13168,"mutex_wait_us":38,"thread_start_us":10625,"threads_started":1}
I20260808 11:42:31.926258 21877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:31.904874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55502 (local address 127.24.76.66:39985)
0808 11:42:31.916535 (+ 11661us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:31.916544 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:31.917956 (+ 1412us) server_negotiation.cc:407] Connection header received
0808 11:42:31.918063 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:31.918070 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:31.918281 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:31.918469 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:42:31.919049 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:31.920437 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:31.923088 (+ 2651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:31.924370 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:31.924843 (+ 473us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:31.925033 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:31.925769 (+ 736us) server_negotiation.cc:299] Negotiation successful
0808 11:42:31.925953 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11470,"thread_start_us":12178,"threads_started":1}
I20260808 11:42:31.929454 21722 tablet_service.cc:1467] Processing CreateTablet for tablet e82b30f6250f4011ae69bd736aa4e07b (DEFAULT_TABLE table=test-table [id=9bc0f89f391742c5bf93b7ae79c58fc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:42:31.929487 21510 tablet_service.cc:1467] Processing CreateTablet for tablet e82b30f6250f4011ae69bd736aa4e07b (DEFAULT_TABLE table=test-table [id=9bc0f89f391742c5bf93b7ae79c58fc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:42:31.929420 21282 tablet_service.cc:1467] Processing CreateTablet for tablet e82b30f6250f4011ae69bd736aa4e07b (DEFAULT_TABLE table=test-table [id=9bc0f89f391742c5bf93b7ae79c58fc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:42:31.930696 21722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e82b30f6250f4011ae69bd736aa4e07b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:42:31.930769 21510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e82b30f6250f4011ae69bd736aa4e07b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:42:31.930771 21282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e82b30f6250f4011ae69bd736aa4e07b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:42:31.941303 21880 tablet_bootstrap.cc:492] T e82b30f6250f4011ae69bd736aa4e07b P 9d54c660bc2a44c0aeb2f4a2849ff955: Bootstrap starting.
I20260808 11:42:31.944659 21880 tablet_bootstrap.cc:654] T e82b30f6250f4011ae69bd736aa4e07b P 9d54c660bc2a44c0aeb2f4a2849ff955: Neither blocks nor log segments found. Creating new log.
I20260808 11:42:31.945609 21880 log.cc:824] T e82b30f6250f4011ae69bd736aa4e07b P 9d54c660bc2a44c0aeb2f4a2849ff955: Log is configured to *not* fsync() on all Append() calls
I20260808 11:42:31.946666 21881 tablet_bootstrap.cc:492] T e82b30f6250f4011ae69bd736aa4e07b P 0525df487545469fb2d53500c2d77270: Bootstrap starting.
I20260808 11:42:31.948357 21882 tablet_bootstrap.cc:492] T e82b30f6250f4011ae69bd736aa4e07b P 024c9160a8a945a5bcec70c056230538: Bootstrap starting.
I20260808 11:42:31.949860 21880 tablet_bootstrap.cc:492] T e82b30f6250f4011ae69bd736aa4e07b P 9d54c660bc2a44c0aeb2f4a2849ff955: No bootstrap required, opened a new log
I20260808 11:42:31.950163 21880 ts_tablet_manager.cc:1397] T e82b30f6250f4011ae69bd736aa4e07b P 9d54c660bc2a44c0aeb2f4a2849ff955: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260808 11:42:31.950213 21881 tablet_bootstrap.cc:654] T e82b30f6250f4011ae69bd736aa4e07b P 0525df487545469fb2d53500c2d77270: Neither blocks nor log segments found. Creating new log.
I20260808 11:42:31.951455 21881 log.cc:824] T e82b30f6250f4011ae69bd736aa4e07b P 0525df487545469fb2d53500c2d77270: Log is configured to *not* fsync() on all Append() calls
I20260808 11:42:31.951726 21882 tablet_bootstrap.cc:654] T e82b30f6250f4011ae69bd736aa4e07b P 024c9160a8a945a5bcec70c056230538: Neither blocks nor log segments found. Creating new log.
I20260808 11:42:31.952700 21882 log.cc:824] T e82b30f6250f4011ae69bd736aa4e07b P 024c9160a8a945a5bcec70c056230538: Log is configured to *not* fsync() on all Append() calls
I20260808 11:42:31.954610 21881 tablet_bootstrap.cc:492] T e82b30f6250f4011ae69bd736aa4e07b P 0525df487545469fb2d53500c2d77270: No bootstrap required, opened a new log
I20260808 11:42:31.954895 21881 ts_tablet_manager.cc:1397] T e82b30f6250f4011ae69bd736aa4e07b P 0525df487545469fb2d53500c2d77270: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260808 11:42:31.958330 21880 raft_consensus.cc:348] T e82b30f6250f4011ae69bd736aa4e07b P 9d54c660bc2a44c0aeb2f4a2849ff955 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d54c660bc2a44c0aeb2f4a2849ff955" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44209 } } peers { permanent_uuid: "024c9160a8a945a5bcec70c056230538" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44363 } } peers { permanent_uuid: "0525df487545469fb2d53500c2d77270" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 39985 } }
I20260808 11:42:31.958652 21880 raft_consensus.cc:374] T e82b30f6250f4011ae69bd736aa4e07b P 9d54c660bc2a44c0aeb2f4a2849ff955 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:42:31.958745 21880 raft_consensus.cc:729] T e82b30f6250f4011ae69bd736aa4e07b P 9d54c660bc2a44c0aeb2f4a2849ff955 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d54c660bc2a44c0aeb2f4a2849ff955, State: Initialized, Role: FOLLOWER
I20260808 11:42:31.959280 21880 consensus_queue.cc:260] T e82b30f6250f4011ae69bd736aa4e07b P 9d54c660bc2a44c0aeb2f4a2849ff955 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d54c660bc2a44c0aeb2f4a2849ff955" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44209 } } peers { permanent_uuid: "024c9160a8a945a5bcec70c056230538" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44363 } } peers { permanent_uuid: "0525df487545469fb2d53500c2d77270" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 39985 } }
I20260808 11:42:31.960264 21848 heartbeater.cc:503] Master 127.24.76.126:42265 was elected leader, sending a full tablet report...
I20260808 11:42:31.960744 21880 ts_tablet_manager.cc:1428] T e82b30f6250f4011ae69bd736aa4e07b P 9d54c660bc2a44c0aeb2f4a2849ff955: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260808 11:42:31.962760 21882 tablet_bootstrap.cc:492] T e82b30f6250f4011ae69bd736aa4e07b P 024c9160a8a945a5bcec70c056230538: No bootstrap required, opened a new log
I20260808 11:42:31.963007 21882 ts_tablet_manager.cc:1397] T e82b30f6250f4011ae69bd736aa4e07b P 024c9160a8a945a5bcec70c056230538: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.004s
I20260808 11:42:31.964618 21881 raft_consensus.cc:348] T e82b30f6250f4011ae69bd736aa4e07b P 0525df487545469fb2d53500c2d77270 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d54c660bc2a44c0aeb2f4a2849ff955" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44209 } } peers { permanent_uuid: "024c9160a8a945a5bcec70c056230538" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44363 } } peers { permanent_uuid: "0525df487545469fb2d53500c2d77270" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 39985 } }
I20260808 11:42:31.964934 21881 raft_consensus.cc:374] T e82b30f6250f4011ae69bd736aa4e07b P 0525df487545469fb2d53500c2d77270 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:42:31.965046 21881 raft_consensus.cc:729] T e82b30f6250f4011ae69bd736aa4e07b P 0525df487545469fb2d53500c2d77270 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0525df487545469fb2d53500c2d77270, State: Initialized, Role: FOLLOWER
I20260808 11:42:31.965662 21881 consensus_queue.cc:260] T e82b30f6250f4011ae69bd736aa4e07b P 0525df487545469fb2d53500c2d77270 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d54c660bc2a44c0aeb2f4a2849ff955" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44209 } } peers { permanent_uuid: "024c9160a8a945a5bcec70c056230538" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44363 } } peers { permanent_uuid: "0525df487545469fb2d53500c2d77270" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 39985 } }
I20260808 11:42:31.967408 21881 ts_tablet_manager.cc:1428] T e82b30f6250f4011ae69bd736aa4e07b P 0525df487545469fb2d53500c2d77270: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260808 11:42:31.967572 21618 heartbeater.cc:503] Master 127.24.76.126:42265 was elected leader, sending a full tablet report...
I20260808 11:42:31.971910 21882 raft_consensus.cc:348] T e82b30f6250f4011ae69bd736aa4e07b P 024c9160a8a945a5bcec70c056230538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d54c660bc2a44c0aeb2f4a2849ff955" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44209 } } peers { permanent_uuid: "024c9160a8a945a5bcec70c056230538" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44363 } } peers { permanent_uuid: "0525df487545469fb2d53500c2d77270" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 39985 } }
I20260808 11:42:31.972218 21882 raft_consensus.cc:374] T e82b30f6250f4011ae69bd736aa4e07b P 024c9160a8a945a5bcec70c056230538 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:42:31.972287 21882 raft_consensus.cc:729] T e82b30f6250f4011ae69bd736aa4e07b P 024c9160a8a945a5bcec70c056230538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 024c9160a8a945a5bcec70c056230538, State: Initialized, Role: FOLLOWER
I20260808 11:42:31.972746 21882 consensus_queue.cc:260] T e82b30f6250f4011ae69bd736aa4e07b P 024c9160a8a945a5bcec70c056230538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d54c660bc2a44c0aeb2f4a2849ff955" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44209 } } peers { permanent_uuid: "024c9160a8a945a5bcec70c056230538" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44363 } } peers { permanent_uuid: "0525df487545469fb2d53500c2d77270" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 39985 } }
I20260808 11:42:31.974171 21882 ts_tablet_manager.cc:1428] T e82b30f6250f4011ae69bd736aa4e07b P 024c9160a8a945a5bcec70c056230538: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260808 11:42:32.010949 21890 raft_consensus.cc:482] T e82b30f6250f4011ae69bd736aa4e07b P 0525df487545469fb2d53500c2d77270 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:42:32.011227 21890 raft_consensus.cc:504] T e82b30f6250f4011ae69bd736aa4e07b P 0525df487545469fb2d53500c2d77270 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d54c660bc2a44c0aeb2f4a2849ff955" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44209 } } peers { permanent_uuid: "024c9160a8a945a5bcec70c056230538" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44363 } } peers { permanent_uuid: "0525df487545469fb2d53500c2d77270" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 39985 } }
I20260808 11:42:32.012838 21890 leader_election.cc:288] T e82b30f6250f4011ae69bd736aa4e07b P 0525df487545469fb2d53500c2d77270 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d54c660bc2a44c0aeb2f4a2849ff955 (127.24.76.67:44209), 024c9160a8a945a5bcec70c056230538 (127.24.76.65:44363)
I20260808 11:42:32.019718 21876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:32.013236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:34557 (local address 127.24.76.65:44363)
0808 11:42:32.013432 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:32.013439 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:32.014186 (+ 747us) server_negotiation.cc:407] Connection header received
0808 11:42:32.014360 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:32.014364 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:32.014423 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:32.014504 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:42:32.015131 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:32.015825 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:32.018086 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:32.018849 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:32.019001 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:32.019073 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:32.019355 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 11:42:32.019476 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 11:42:32.019780 21872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:32.013221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:40941 (local address 127.24.76.67:44209)
0808 11:42:32.013424 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:32.013430 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:32.014224 (+ 794us) server_negotiation.cc:407] Connection header received
0808 11:42:32.014384 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:32.014388 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:32.014447 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:32.014517 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:42:32.015184 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:32.015894 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:32.018130 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:32.018892 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:32.019034 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:32.019107 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:32.019416 (+ 309us) server_negotiation.cc:299] Negotiation successful
0808 11:42:32.019543 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 11:42:32.019874 21893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:32.013179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:44363 (local address 127.24.76.66:34557)
0808 11:42:32.014077 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:32.014093 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:32.014230 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:32.014522 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:32.014526 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:32.014549 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:42:32.014987 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:32.015001 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:32.015944 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:32.015949 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:32.017925 (+ 1976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:32.017940 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:32.019078 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:32.019085 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:32.019208 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:32.019234 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 11:42:32.019314 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 11:42:32.019430 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"mutex_wait_us":18,"thread_start_us":428,"threads_started":1}
I20260808 11:42:32.019897 21895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:32.013254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:44209 (local address 127.24.76.66:40941)
0808 11:42:32.014115 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:32.014130 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:32.014252 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:32.014545 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:32.014549 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:32.014572 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:42:32.015032 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:32.015041 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:32.016029 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:32.016033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:32.017982 (+ 1949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:32.017995 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:32.019114 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:32.019121 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:32.019272 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:32.019294 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 11:42:32.019369 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 11:42:32.019481 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":762,"mutex_wait_us":71,"thread_start_us":469,"threads_started":1}
I20260808 11:42:32.020797 21317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e82b30f6250f4011ae69bd736aa4e07b" candidate_uuid: "0525df487545469fb2d53500c2d77270" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "024c9160a8a945a5bcec70c056230538" is_pre_election: true
I20260808 11:42:32.020887 21753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e82b30f6250f4011ae69bd736aa4e07b" candidate_uuid: "0525df487545469fb2d53500c2d77270" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d54c660bc2a44c0aeb2f4a2849ff955" is_pre_election: true
I20260808 11:42:32.021282 21753 raft_consensus.cc:2454] T e82b30f6250f4011ae69bd736aa4e07b P 9d54c660bc2a44c0aeb2f4a2849ff955 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0525df487545469fb2d53500c2d77270 in term 0.
I20260808 11:42:32.021287 21317 raft_consensus.cc:2454] T e82b30f6250f4011ae69bd736aa4e07b P 024c9160a8a945a5bcec70c056230538 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0525df487545469fb2d53500c2d77270 in term 0.
I20260808 11:42:32.021847 21429 leader_election.cc:302] T e82b30f6250f4011ae69bd736aa4e07b P 0525df487545469fb2d53500c2d77270 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0525df487545469fb2d53500c2d77270, 9d54c660bc2a44c0aeb2f4a2849ff955; no voters:
I20260808 11:42:32.022253 21890 raft_consensus.cc:2781] T e82b30f6250f4011ae69bd736aa4e07b P 0525df487545469fb2d53500c2d77270 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:42:32.022367 21890 raft_consensus.cc:482] T e82b30f6250f4011ae69bd736aa4e07b P 0525df487545469fb2d53500c2d77270 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:42:32.022437 21890 raft_consensus.cc:3037] T e82b30f6250f4011ae69bd736aa4e07b P 0525df487545469fb2d53500c2d77270 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:42:32.024507 21890 raft_consensus.cc:504] T e82b30f6250f4011ae69bd736aa4e07b P 0525df487545469fb2d53500c2d77270 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d54c660bc2a44c0aeb2f4a2849ff955" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44209 } } peers { permanent_uuid: "024c9160a8a945a5bcec70c056230538" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44363 } } peers { permanent_uuid: "0525df487545469fb2d53500c2d77270" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 39985 } }
I20260808 11:42:32.025053 21890 leader_election.cc:288] T e82b30f6250f4011ae69bd736aa4e07b P 0525df487545469fb2d53500c2d77270 [CANDIDATE]: Term 1 election: Requested vote from peers 9d54c660bc2a44c0aeb2f4a2849ff955 (127.24.76.67:44209), 024c9160a8a945a5bcec70c056230538 (127.24.76.65:44363)
I20260808 11:42:32.025398 21753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e82b30f6250f4011ae69bd736aa4e07b" candidate_uuid: "0525df487545469fb2d53500c2d77270" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d54c660bc2a44c0aeb2f4a2849ff955"
I20260808 11:42:32.025521 21317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e82b30f6250f4011ae69bd736aa4e07b" candidate_uuid: "0525df487545469fb2d53500c2d77270" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "024c9160a8a945a5bcec70c056230538"
I20260808 11:42:32.025607 21753 raft_consensus.cc:3037] T e82b30f6250f4011ae69bd736aa4e07b P 9d54c660bc2a44c0aeb2f4a2849ff955 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:42:32.025691 21317 raft_consensus.cc:3037] T e82b30f6250f4011ae69bd736aa4e07b P 024c9160a8a945a5bcec70c056230538 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:42:32.028760 21317 raft_consensus.cc:2454] T e82b30f6250f4011ae69bd736aa4e07b P 024c9160a8a945a5bcec70c056230538 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0525df487545469fb2d53500c2d77270 in term 1.
I20260808 11:42:32.028760 21753 raft_consensus.cc:2454] T e82b30f6250f4011ae69bd736aa4e07b P 9d54c660bc2a44c0aeb2f4a2849ff955 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0525df487545469fb2d53500c2d77270 in term 1.
I20260808 11:42:32.029137 21431 leader_election.cc:302] T e82b30f6250f4011ae69bd736aa4e07b P 0525df487545469fb2d53500c2d77270 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 024c9160a8a945a5bcec70c056230538, 0525df487545469fb2d53500c2d77270; no voters:
I20260808 11:42:32.029408 21890 raft_consensus.cc:2781] T e82b30f6250f4011ae69bd736aa4e07b P 0525df487545469fb2d53500c2d77270 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:42:32.029808 21890 raft_consensus.cc:686] T e82b30f6250f4011ae69bd736aa4e07b P 0525df487545469fb2d53500c2d77270 [term 1 LEADER]: Becoming Leader. State: Replica: 0525df487545469fb2d53500c2d77270, State: Running, Role: LEADER
I20260808 11:42:32.030328 21890 consensus_queue.cc:237] T e82b30f6250f4011ae69bd736aa4e07b P 0525df487545469fb2d53500c2d77270 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d54c660bc2a44c0aeb2f4a2849ff955" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44209 } } peers { permanent_uuid: "024c9160a8a945a5bcec70c056230538" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44363 } } peers { permanent_uuid: "0525df487545469fb2d53500c2d77270" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 39985 } }
I20260808 11:42:32.034391 21080 catalog_manager.cc:5495] T e82b30f6250f4011ae69bd736aa4e07b P 0525df487545469fb2d53500c2d77270 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0525df487545469fb2d53500c2d77270 (127.24.76.66). New cstate: current_term: 1 leader_uuid: "0525df487545469fb2d53500c2d77270" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d54c660bc2a44c0aeb2f4a2849ff955" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "024c9160a8a945a5bcec70c056230538" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0525df487545469fb2d53500c2d77270" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 39985 } health_report { overall_health: HEALTHY } } }
W20260808 11:42:32.055274 21396 tablet.cc:2369] T e82b30f6250f4011ae69bd736aa4e07b P 024c9160a8a945a5bcec70c056230538: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:42:32.066848 21877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:32.061939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55508 (local address 127.24.76.66:39985)
0808 11:42:32.062156 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:32.062161 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:32.062178 (+ 17us) server_negotiation.cc:407] Connection header received
0808 11:42:32.062361 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:32.062365 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:32.062448 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:32.062566 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:42:32.062872 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:32.063563 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:32.064814 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:32.065196 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:32.065273 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:32.065740 (+ 467us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:42:32.066262 (+ 522us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:42:32.066391 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:32.066522 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 11:42:32.066564 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 11:42:32.075232 21317 raft_consensus.cc:1261] T e82b30f6250f4011ae69bd736aa4e07b P 024c9160a8a945a5bcec70c056230538 [term 1 FOLLOWER]: Refusing update from remote peer 0525df487545469fb2d53500c2d77270: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:42:32.075263 21753 raft_consensus.cc:1261] T e82b30f6250f4011ae69bd736aa4e07b P 9d54c660bc2a44c0aeb2f4a2849ff955 [term 1 FOLLOWER]: Refusing update from remote peer 0525df487545469fb2d53500c2d77270: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:42:32.076176 21898 consensus_queue.cc:1035] T e82b30f6250f4011ae69bd736aa4e07b P 0525df487545469fb2d53500c2d77270 [LEADER]: Connected to new peer: Peer: permanent_uuid: "024c9160a8a945a5bcec70c056230538" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:42:32.076538 21890 consensus_queue.cc:1035] T e82b30f6250f4011ae69bd736aa4e07b P 0525df487545469fb2d53500c2d77270 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d54c660bc2a44c0aeb2f4a2849ff955" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 44209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 11:42:32.078117 21619 tablet.cc:2369] T e82b30f6250f4011ae69bd736aa4e07b P 0525df487545469fb2d53500c2d77270: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:42:32.087970 21903 mvcc.cc:205] Tried to move back new op lower bound from 7316231586088951808 to 7316231585919725568. Current Snapshot: MvccSnapshot[applied={T|T < 7316231586088951808}]
I20260808 11:42:32.088627 21905 mvcc.cc:205] Tried to move back new op lower bound from 7316231586088951808 to 7316231585919725568. Current Snapshot: MvccSnapshot[applied={T|T < 7316231586088951808}]
I20260808 11:42:32.101475 21079 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:42:32.101723 21079 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:42:32.105315 21079 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1b187010b22f4f94a964526ed0d1f71e: Sending DeleteTablet for 3 replicas of tablet e82b30f6250f4011ae69bd736aa4e07b
I20260808 11:42:32.106382 21510 tablet_service.cc:1514] Processing DeleteTablet for tablet e82b30f6250f4011ae69bd736aa4e07b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:42:32 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:55502
I20260808 11:42:32.106375 21282 tablet_service.cc:1514] Processing DeleteTablet for tablet e82b30f6250f4011ae69bd736aa4e07b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:42:32 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:44714
I20260808 11:42:32.106395 21722 tablet_service.cc:1514] Processing DeleteTablet for tablet e82b30f6250f4011ae69bd736aa4e07b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:42:32 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:37686
I20260808 11:42:32.110212 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 21189
I20260808 11:42:32.121126 21917 tablet_replica.cc:330] T e82b30f6250f4011ae69bd736aa4e07b P 0525df487545469fb2d53500c2d77270: stopping tablet replica
I20260808 11:42:32.121731 21917 raft_consensus.cc:2229] T e82b30f6250f4011ae69bd736aa4e07b P 0525df487545469fb2d53500c2d77270 [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:42:32.125874 21917 raft_consensus.cc:2258] T e82b30f6250f4011ae69bd736aa4e07b P 0525df487545469fb2d53500c2d77270 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:42:32.129606 21917 ts_tablet_manager.cc:1905] T e82b30f6250f4011ae69bd736aa4e07b P 0525df487545469fb2d53500c2d77270: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 08 11:42:32 dist-test-slave-tc2s krb5kdc[21002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.65@KRBTEST.COM
I20260808 11:42:32.142365 21916 tablet_replica.cc:330] T e82b30f6250f4011ae69bd736aa4e07b P 024c9160a8a945a5bcec70c056230538: stopping tablet replica
I20260808 11:42:32.142728 21916 raft_consensus.cc:2229] T e82b30f6250f4011ae69bd736aa4e07b P 024c9160a8a945a5bcec70c056230538 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:42:32.143180 21916 raft_consensus.cc:2258] T e82b30f6250f4011ae69bd736aa4e07b P 024c9160a8a945a5bcec70c056230538 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:42:32.143855 21918 tablet_replica.cc:330] T e82b30f6250f4011ae69bd736aa4e07b P 9d54c660bc2a44c0aeb2f4a2849ff955: stopping tablet replica
I20260808 11:42:32.144186 21918 raft_consensus.cc:2229] T e82b30f6250f4011ae69bd736aa4e07b P 9d54c660bc2a44c0aeb2f4a2849ff955 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:42:32.144563 21918 raft_consensus.cc:2258] T e82b30f6250f4011ae69bd736aa4e07b P 9d54c660bc2a44c0aeb2f4a2849ff955 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:42:32.146955 21918 ts_tablet_manager.cc:1905] T e82b30f6250f4011ae69bd736aa4e07b P 9d54c660bc2a44c0aeb2f4a2849ff955: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:42:32.147516 21916 ts_tablet_manager.cc:1905] T e82b30f6250f4011ae69bd736aa4e07b P 024c9160a8a945a5bcec70c056230538: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:42:32.153605 21918 ts_tablet_manager.cc:1918] T e82b30f6250f4011ae69bd736aa4e07b P 9d54c660bc2a44c0aeb2f4a2849ff955: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:42:32.153815 21918 log.cc:1196] T e82b30f6250f4011ae69bd736aa4e07b P 9d54c660bc2a44c0aeb2f4a2849ff955: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-2/wal/wals/e82b30f6250f4011ae69bd736aa4e07b
I20260808 11:42:32.154536 21876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:32.115251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44716 (local address 127.24.76.65:44363)
0808 11:42:32.115463 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:32.115471 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:32.117173 (+ 1702us) server_negotiation.cc:407] Connection header received
0808 11:42:32.117381 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:32.117387 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:32.117516 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:32.117626 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:32.122089 (+ 4463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:32.123392 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:32.127150 (+ 3758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:32.127835 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:32.127939 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:32.146019 (+ 18080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:32.146036 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:32.146055 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:42:32.146105 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:32.150123 (+ 4018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:32.151113 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:32.151122 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:32.151126 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:32.151306 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:32.151814 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:32.151824 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:32.151827 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:32.152336 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:32.152637 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:32.153914 (+ 1277us) server_negotiation.cc:299] Negotiation successful
0808 11:42:32.154166 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 11:42:32.154712 21917 ts_tablet_manager.cc:1918] T e82b30f6250f4011ae69bd736aa4e07b P 0525df487545469fb2d53500c2d77270: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:42:32.154932 21917 log.cc:1196] T e82b30f6250f4011ae69bd736aa4e07b P 0525df487545469fb2d53500c2d77270: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-1/wal/wals/e82b30f6250f4011ae69bd736aa4e07b
I20260808 11:42:32.155436 21918 ts_tablet_manager.cc:1939] T e82b30f6250f4011ae69bd736aa4e07b P 9d54c660bc2a44c0aeb2f4a2849ff955: Deleting consensus metadata
I20260808 11:42:32.155453 21389 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:42:32.157196 21056 catalog_manager.cc:4841] TS 9d54c660bc2a44c0aeb2f4a2849ff955 (127.24.76.67:44209): tablet e82b30f6250f4011ae69bd736aa4e07b (table test-table [id=9bc0f89f391742c5bf93b7ae79c58fc8]) successfully deleted
I20260808 11:42:32.162426 21917 ts_tablet_manager.cc:1939] T e82b30f6250f4011ae69bd736aa4e07b P 0525df487545469fb2d53500c2d77270: Deleting consensus metadata
I20260808 11:42:32.164120 21062 catalog_manager.cc:4841] TS 0525df487545469fb2d53500c2d77270 (127.24.76.66:39985): tablet e82b30f6250f4011ae69bd736aa4e07b (table test-table [id=9bc0f89f391742c5bf93b7ae79c58fc8]) successfully deleted
I20260808 11:42:32.258234 21916 ts_tablet_manager.cc:1918] T e82b30f6250f4011ae69bd736aa4e07b P 024c9160a8a945a5bcec70c056230538: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:42:32.258522 21916 log.cc:1196] T e82b30f6250f4011ae69bd736aa4e07b P 024c9160a8a945a5bcec70c056230538: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786189042529715-24881-0/minicluster-data/ts-0/wal/wals/e82b30f6250f4011ae69bd736aa4e07b
I20260808 11:42:32.258937 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 21189
W20260808 11:42:32.278406 21061 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.76.65:44363 (error 108)
I20260808 11:42:32.279145 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 21406
I20260808 11:42:32.280602 21871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:32.280105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:44363 (local address 127.0.0.1:44728)
0808 11:42:32.280275 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:32.280426 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:44363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 11:42:32.280901 21061 catalog_manager.cc:4584] TS 024c9160a8a945a5bcec70c056230538 (127.24.76.65:44363): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e82b30f6250f4011ae69bd736aa4e07b: Network error: Client connection negotiation failed: client connection to 127.24.76.65:44363: connect: Connection refused (error 111)
Aug 08 11:42:32 dist-test-slave-tc2s krb5kdc[21002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.66@KRBTEST.COM
I20260808 11:42:32.296583 21877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:32.279993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55524 (local address 127.24.76.66:39985)
0808 11:42:32.280397 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:32.280403 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:32.280425 (+ 22us) server_negotiation.cc:407] Connection header received
0808 11:42:32.280495 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:32.280500 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:32.280584 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:32.280693 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:32.282391 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:32.283424 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:32.285214 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:32.285666 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:32.285764 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:32.290756 (+ 4992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:32.290769 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:32.290784 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:42:32.290820 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:32.293746 (+ 2926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:32.294490 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:32.294497 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:32.294499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:32.294641 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:32.295045 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:32.295050 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:32.295052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:32.295412 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:32.295747 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:32.296043 (+ 296us) server_negotiation.cc:299] Negotiation successful
0808 11:42:32.296253 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269}
I20260808 11:42:32.297287 21611 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:42:32.334689 21871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:32.334110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:44363 (local address 127.0.0.1:44736)
0808 11:42:32.334371 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:32.334490 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:44363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260808 11:42:32.335717 21871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:32.335328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:44363 (local address 127.0.0.1:44750)
0808 11:42:32.335480 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:32.335565 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:44363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 11:42:32.396050 21871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:32.395515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:44363 (local address 127.0.0.1:44758)
0808 11:42:32.395738 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:32.395849 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:44363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260808 11:42:32.397059 21871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:32.396697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:44363 (local address 127.0.0.1:44770)
0808 11:42:32.396831 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:32.396914 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:44363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 11:42:32.412084 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 21406
I20260808 11:42:32.440299 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 21627
Aug 08 11:42:32 dist-test-slave-tc2s krb5kdc[21002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.67@KRBTEST.COM
I20260808 11:42:32.454352 21872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:32.440999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37692 (local address 127.24.76.67:44209)
0808 11:42:32.441196 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:32.441201 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:32.441220 (+ 19us) server_negotiation.cc:407] Connection header received
0808 11:42:32.441354 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:32.441358 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:32.441422 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:32.441510 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:32.442872 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:32.443595 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:32.444793 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:32.445159 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:32.445244 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:32.449328 (+ 4084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:32.449340 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:32.449356 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:42:32.449391 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:32.451884 (+ 2493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:32.452515 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:32.452523 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:32.452525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:32.452690 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:32.453009 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:32.453011 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:32.453013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:32.453363 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:32.453574 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:32.453810 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 11:42:32.454030 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 11:42:32.455008 21837 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:42:32.477783 21871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:32.477312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:44363 (local address 127.0.0.1:44786)
0808 11:42:32.477528 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:32.477627 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:44363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260808 11:42:32.478833 21871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:32.478450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:44363 (local address 127.0.0.1:44802)
0808 11:42:32.478635 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:32.478706 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:44363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260808 11:42:32.542464 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 21627
I20260808 11:42:32.561933 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 21029
I20260808 11:42:32.563020 21136 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:42:32.675203 21871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:32.674431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:44363 (local address 127.0.0.1:36006)
0808 11:42:32.674637 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:32.674762 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:44363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 11:42:32.675473 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 21029
2026-08-08T11:42:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4470 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T11:42:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:42:32Z Disabled control of system clock
I20260808 11:42:32.769884 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:34517
--webserver_interface=127.24.76.126
--webserver_port=0
--builtin_ntp_servers=127.24.76.84:37901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:34517
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 11:42:33.096187 21932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:42:33.096496 21932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:42:33.096624 21932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:42:33.103753 21932 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:42:33.103855 21932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:42:33.103920 21932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:42:33.103998 21932 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:42:33.104055 21932 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:42:33.113008 21932 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:37901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:34517
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:34517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:42:33.114318 21932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:42:33.116084 21932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:42:33.127555 21943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:33.127586 21946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:33.127513 21940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:42:33.128693 21932 server_base.cc:1034] running on GCE node
I20260808 11:42:33.129431 21932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:42:33.130810 21932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:42:33.132012 21932 hybrid_clock.cc:648] HybridClock initialized: now 1786189353131968 us; error 34 us; skew 500 ppm
I20260808 11:42:33.132335 21932 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:42:33.139659 21932 webserver.cc:460] Webserver started at http://127.24.76.126:38439/ using document root <none> and password file <none>
I20260808 11:42:33.140355 21932 fs_manager.cc:359] Metadata directory not provided
I20260808 11:42:33.140468 21932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:42:33.140777 21932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:42:33.143746 21932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "0b081a89d3e647258a8bbdb7b78dab8d"
format_stamp: "Formatted at 2026-08-08 11:42:33 on dist-test-slave-tc2s"
server_key: "2f69716930fb63491c3036122eb339aa"
server_key_iv: "bd460346997c26db18e6c4680cfa3ccd"
server_key_version: "encryptionkey@0"
I20260808 11:42:33.144568 21932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "0b081a89d3e647258a8bbdb7b78dab8d"
format_stamp: "Formatted at 2026-08-08 11:42:33 on dist-test-slave-tc2s"
server_key: "2f69716930fb63491c3036122eb339aa"
server_key_iv: "bd460346997c26db18e6c4680cfa3ccd"
server_key_version: "encryptionkey@0"
I20260808 11:42:33.149662 21932 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 11:42:33.153044 21955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:33.154309 21932 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:42:33.154484 21932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "0b081a89d3e647258a8bbdb7b78dab8d"
format_stamp: "Formatted at 2026-08-08 11:42:33 on dist-test-slave-tc2s"
server_key: "2f69716930fb63491c3036122eb339aa"
server_key_iv: "bd460346997c26db18e6c4680cfa3ccd"
server_key_version: "encryptionkey@0"
I20260808 11:42:33.154685 21932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:42:33.175323 21932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:42:33.176090 21932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:42:33.176342 21932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:42:33.193374 21932 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:34517
I20260808 11:42:33.193437 22053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:34517 every 8 connection(s)
I20260808 11:42:33.194595 21932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:42:33.198171 22056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:42:33.202975 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 21932
I20260808 11:42:33.203166 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:42:33.203752 24881 external_mini_cluster.cc:1428] Setting key 05435b431ad14963361a1c3804991380
I20260808 11:42:33.210419 22056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b081a89d3e647258a8bbdb7b78dab8d: Bootstrap starting.
I20260808 11:42:33.215353 22056 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b081a89d3e647258a8bbdb7b78dab8d: Neither blocks nor log segments found. Creating new log.
I20260808 11:42:33.216758 22056 log.cc:824] T 00000000000000000000000000000000 P 0b081a89d3e647258a8bbdb7b78dab8d: Log is configured to *not* fsync() on all Append() calls
I20260808 11:42:33.220382 22056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b081a89d3e647258a8bbdb7b78dab8d: No bootstrap required, opened a new log
I20260808 11:42:33.229230 22059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:33.205763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50996 (local address 127.24.76.126:34517)
0808 11:42:33.216974 (+ 11211us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:33.216987 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:33.217039 (+ 52us) server_negotiation.cc:407] Connection header received
0808 11:42:33.219426 (+ 2387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:33.219454 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:33.219937 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:33.220391 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:33.220950 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:33.222985 (+ 2035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:33.224254 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:33.225358 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:33.225554 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:33.226522 (+ 968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:33.226531 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:33.226549 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:42:33.227242 (+ 693us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:33.227358 (+ 116us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:42:33.227389 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:33.227497 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:33.227648 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 11:42:33.227982 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10682,"thread_start_us":11237,"threads_started":1}
I20260808 11:42:33.230152 22056 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b081a89d3e647258a8bbdb7b78dab8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b081a89d3e647258a8bbdb7b78dab8d" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 34517 } }
I20260808 11:42:33.230494 22056 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b081a89d3e647258a8bbdb7b78dab8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:42:33.230595 22056 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b081a89d3e647258a8bbdb7b78dab8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b081a89d3e647258a8bbdb7b78dab8d, State: Initialized, Role: FOLLOWER
I20260808 11:42:33.231179 22056 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b081a89d3e647258a8bbdb7b78dab8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b081a89d3e647258a8bbdb7b78dab8d" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 34517 } }
I20260808 11:42:33.231527 22056 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b081a89d3e647258a8bbdb7b78dab8d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:42:33.231670 22056 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b081a89d3e647258a8bbdb7b78dab8d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:42:33.231840 22056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b081a89d3e647258a8bbdb7b78dab8d [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:42:33.234088 22056 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b081a89d3e647258a8bbdb7b78dab8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b081a89d3e647258a8bbdb7b78dab8d" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 34517 } }
I20260808 11:42:33.234509 22056 leader_election.cc:302] T 00000000000000000000000000000000 P 0b081a89d3e647258a8bbdb7b78dab8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b081a89d3e647258a8bbdb7b78dab8d; no voters:
I20260808 11:42:33.235019 22056 leader_election.cc:288] T 00000000000000000000000000000000 P 0b081a89d3e647258a8bbdb7b78dab8d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:42:33.235143 22061 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b081a89d3e647258a8bbdb7b78dab8d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:42:33.235767 22061 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b081a89d3e647258a8bbdb7b78dab8d [term 1 LEADER]: Becoming Leader. State: Replica: 0b081a89d3e647258a8bbdb7b78dab8d, State: Running, Role: LEADER
I20260808 11:42:33.236292 22061 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b081a89d3e647258a8bbdb7b78dab8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b081a89d3e647258a8bbdb7b78dab8d" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 34517 } }
I20260808 11:42:33.236868 22056 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b081a89d3e647258a8bbdb7b78dab8d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:42:33.238667 22062 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b081a89d3e647258a8bbdb7b78dab8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b081a89d3e647258a8bbdb7b78dab8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b081a89d3e647258a8bbdb7b78dab8d" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 34517 } } }
I20260808 11:42:33.238965 22062 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b081a89d3e647258a8bbdb7b78dab8d [sys.catalog]: This master's current role is: LEADER
I20260808 11:42:33.240545 22067 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:42:33.240762 22063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b081a89d3e647258a8bbdb7b78dab8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b081a89d3e647258a8bbdb7b78dab8d. Latest consensus state: current_term: 1 leader_uuid: "0b081a89d3e647258a8bbdb7b78dab8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b081a89d3e647258a8bbdb7b78dab8d" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 34517 } } }
I20260808 11:42:33.241084 22063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b081a89d3e647258a8bbdb7b78dab8d [sys.catalog]: This master's current role is: LEADER
I20260808 11:42:33.245684 22067 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:42:33.254521 22067 catalog_manager.cc:1347] Generated new cluster ID: c6be6c806a744b65ae2e5bc5d93eba8b
I20260808 11:42:33.254643 22067 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:42:33.264595 22067 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:42:33.266283 22067 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:42:33.281160 22067 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b081a89d3e647258a8bbdb7b78dab8d: Generated new TSK 0
I20260808 11:42:33.282094 22067 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:42:33.293494 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.65:0
--local_ip_for_outbound_sockets=127.24.76.65
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:34517
--builtin_ntp_servers=127.24.76.84:37901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 11:42:33.629391 22095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:42:33.629647 22095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:42:33.629766 22095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:42:33.636689 22095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:42:33.636794 22095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:42:33.636942 22095 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.65
I20260808 11:42:33.645792 22095 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:37901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:34517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.65
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:42:33.646960 22095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:42:33.648656 22095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:42:33.661084 22101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:33.661123 22108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:33.661046 22102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:42:33.661834 22095 server_base.cc:1034] running on GCE node
I20260808 11:42:33.662498 22095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:42:33.663381 22095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:42:33.664577 22095 hybrid_clock.cc:648] HybridClock initialized: now 1786189353664524 us; error 50 us; skew 500 ppm
I20260808 11:42:33.664866 22095 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:42:33.667014 22095 webserver.cc:460] Webserver started at http://127.24.76.65:44717/ using document root <none> and password file <none>
I20260808 11:42:33.667652 22095 fs_manager.cc:359] Metadata directory not provided
I20260808 11:42:33.667752 22095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:42:33.668069 22095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:42:33.670847 22095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-0/data/instance:
uuid: "b41557f51bbf4627ae4efaa6f40cbe28"
format_stamp: "Formatted at 2026-08-08 11:42:33 on dist-test-slave-tc2s"
server_key: "0f6750b40ca5c205cf1969c2ed75effa"
server_key_iv: "8308a04eda44f7a2f9f0a05f82dea2bd"
server_key_version: "encryptionkey@0"
I20260808 11:42:33.671615 22095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance:
uuid: "b41557f51bbf4627ae4efaa6f40cbe28"
format_stamp: "Formatted at 2026-08-08 11:42:33 on dist-test-slave-tc2s"
server_key: "0f6750b40ca5c205cf1969c2ed75effa"
server_key_iv: "8308a04eda44f7a2f9f0a05f82dea2bd"
server_key_version: "encryptionkey@0"
I20260808 11:42:33.676396 22095 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 11:42:33.679678 22117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:33.680853 22095 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 11:42:33.680998 22095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-0/wal
uuid: "b41557f51bbf4627ae4efaa6f40cbe28"
format_stamp: "Formatted at 2026-08-08 11:42:33 on dist-test-slave-tc2s"
server_key: "0f6750b40ca5c205cf1969c2ed75effa"
server_key_iv: "8308a04eda44f7a2f9f0a05f82dea2bd"
server_key_version: "encryptionkey@0"
I20260808 11:42:33.681187 22095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:42:33.699966 22095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:42:33.700703 22095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:42:33.700978 22095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:42:33.701848 22095 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:42:33.703246 22095 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:42:33.703341 22095 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:33.703452 22095 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:42:33.703522 22095 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:33.732956 22095 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.65:38039
I20260808 11:42:33.733007 22310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.65:38039 every 8 connection(s)
I20260808 11:42:33.734239 22095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
I20260808 11:42:33.743713 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 22095
I20260808 11:42:33.743875 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance
I20260808 11:42:33.744349 24881 external_mini_cluster.cc:1428] Setting key 254d7a9e268fe82fe53343e8c75fc5d0
I20260808 11:42:33.746502 22316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:33.736571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:40587 (local address 127.24.76.126:34517)
0808 11:42:33.737036 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:33.737041 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:33.738219 (+ 1178us) server_negotiation.cc:407] Connection header received
0808 11:42:33.740064 (+ 1845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:33.740068 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:33.740134 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:33.740227 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:33.741330 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:33.742097 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:33.743710 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:33.744174 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:33.744269 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:33.745693 (+ 1424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:33.745699 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:33.745704 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:42:33.745725 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:33.745772 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:42:33.745787 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:33.745871 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:33.746126 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 11:42:33.746219 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":219,"threads_started":1}
I20260808 11:42:33.747386 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.66:0
--local_ip_for_outbound_sockets=127.24.76.66
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:34517
--builtin_ntp_servers=127.24.76.84:37901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 11:42:33.747638 22317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:33.737157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:34517 (local address 127.24.76.65:40587)
0808 11:42:33.738071 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:33.738123 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:33.739772 (+ 1649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:33.740438 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:33.740446 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:33.740560 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:42:33.741170 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:33.741188 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:33.742243 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:33.742247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:33.743592 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:33.743608 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:33.744386 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:33.744395 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:33.744690 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:33.745376 (+ 686us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:42:33.745379 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:42:33.745418 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:42:33.745505 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:42:33.745874 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:42:33.745880 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:42:33.745885 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 11:42:33.746078 (+ 193us) client_negotiation.cc:240] Negotiation successful
0808 11:42:33.746401 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":260,"threads_started":1}
I20260808 11:42:33.749048 22313 heartbeater.cc:347] Connected to a master server at 127.24.76.126:34517
I20260808 11:42:33.749347 22313 heartbeater.cc:464] Registering TS with master...
I20260808 11:42:33.750190 22313 heartbeater.cc:511] Master 127.24.76.126:34517 requested a full tablet report, sending...
I20260808 11:42:33.752117 21991 ts_manager.cc:194] Registered new tserver with Master: b41557f51bbf4627ae4efaa6f40cbe28 (127.24.76.65:38039)
I20260808 11:42:33.753525 21991 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.76.65:40587
W20260808 11:42:34.074419 22318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:42:34.074699 22318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:42:34.074824 22318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:42:34.082566 22318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:42:34.082662 22318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:42:34.082799 22318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.66
I20260808 11:42:34.091499 22318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:37901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:34517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.66
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:42:34.092653 22318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:42:34.094393 22318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:42:34.106786 22326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:34.106823 22327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:34.106858 22330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:42:34.107766 22318 server_base.cc:1034] running on GCE node
I20260808 11:42:34.108359 22318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:42:34.109202 22318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:42:34.110397 22318 hybrid_clock.cc:648] HybridClock initialized: now 1786189354110360 us; error 24 us; skew 500 ppm
I20260808 11:42:34.110687 22318 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:42:34.112751 22318 webserver.cc:460] Webserver started at http://127.24.76.66:44929/ using document root <none> and password file <none>
I20260808 11:42:34.113391 22318 fs_manager.cc:359] Metadata directory not provided
I20260808 11:42:34.113487 22318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:42:34.113776 22318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:42:34.116463 22318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-1/data/instance:
uuid: "8f5fe547fc8747c586c2637d19f6b947"
format_stamp: "Formatted at 2026-08-08 11:42:34 on dist-test-slave-tc2s"
server_key: "c353c2fd0bfd4f08ccb6396a7eecaf7a"
server_key_iv: "d8424d2a7c75e263f71d07a124a406a6"
server_key_version: "encryptionkey@0"
I20260808 11:42:34.117436 22318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance:
uuid: "8f5fe547fc8747c586c2637d19f6b947"
format_stamp: "Formatted at 2026-08-08 11:42:34 on dist-test-slave-tc2s"
server_key: "c353c2fd0bfd4f08ccb6396a7eecaf7a"
server_key_iv: "d8424d2a7c75e263f71d07a124a406a6"
server_key_version: "encryptionkey@0"
I20260808 11:42:34.122231 22318 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 11:42:34.125349 22337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:34.126561 22318 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 11:42:34.126705 22318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-1/wal
uuid: "8f5fe547fc8747c586c2637d19f6b947"
format_stamp: "Formatted at 2026-08-08 11:42:34 on dist-test-slave-tc2s"
server_key: "c353c2fd0bfd4f08ccb6396a7eecaf7a"
server_key_iv: "d8424d2a7c75e263f71d07a124a406a6"
server_key_version: "encryptionkey@0"
I20260808 11:42:34.126894 22318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:42:34.147662 22318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:42:34.148434 22318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:42:34.148701 22318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:42:34.149590 22318 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:42:34.151002 22318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:42:34.151103 22318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:34.151206 22318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:42:34.151274 22318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:34.178762 22318 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.66:37363
I20260808 11:42:34.178830 22527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.66:37363 every 8 connection(s)
I20260808 11:42:34.180034 22318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
I20260808 11:42:34.187889 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 22318
I20260808 11:42:34.188074 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance
I20260808 11:42:34.188515 24881 external_mini_cluster.cc:1428] Setting key e979e8d721d76522e69c134054c68550
I20260808 11:42:34.191231 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.67:0
--local_ip_for_outbound_sockets=127.24.76.67
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:34517
--builtin_ntp_servers=127.24.76.84:37901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 11:42:34.192137 22316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:34.182577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:40397 (local address 127.24.76.126:34517)
0808 11:42:34.182752 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:34.182757 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:34.184085 (+ 1328us) server_negotiation.cc:407] Connection header received
0808 11:42:34.185814 (+ 1729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:34.185819 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:34.185906 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:34.186001 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:34.186976 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:34.187797 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:34.189493 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:34.189979 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:34.190088 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:34.191337 (+ 1249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:34.191343 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:34.191348 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:42:34.191369 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:34.191408 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:42:34.191422 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:34.191511 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:34.191701 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 11:42:34.191800 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":61}
I20260808 11:42:34.193260 22531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:34.183069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:34517 (local address 127.24.76.66:40397)
0808 11:42:34.183919 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:34.183982 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:34.185583 (+ 1601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:34.186200 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:34.186208 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:34.186301 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:42:34.186831 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:34.186847 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:34.187934 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:34.187938 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:34.189346 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:34.189359 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:34.190200 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:34.190208 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:34.190415 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:34.191046 (+ 631us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:42:34.191049 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:42:34.191086 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:42:34.191162 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:42:34.191513 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:42:34.191524 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:42:34.191533 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 11:42:34.191680 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 11:42:34.192061 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":254,"threads_started":1}
I20260808 11:42:34.194785 22529 heartbeater.cc:347] Connected to a master server at 127.24.76.126:34517
I20260808 11:42:34.195087 22529 heartbeater.cc:464] Registering TS with master...
I20260808 11:42:34.195866 22529 heartbeater.cc:511] Master 127.24.76.126:34517 requested a full tablet report, sending...
I20260808 11:42:34.197395 21991 ts_manager.cc:194] Registered new tserver with Master: 8f5fe547fc8747c586c2637d19f6b947 (127.24.76.66:37363)
I20260808 11:42:34.198438 21991 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.76.66:40397
W20260808 11:42:34.523281 22532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:42:34.523550 22532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:42:34.523644 22532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:42:34.530534 22532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:42:34.530632 22532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:42:34.530764 22532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.67
I20260808 11:42:34.539512 22532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:37901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:34517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.67
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:42:34.540652 22532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:42:34.542567 22532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:42:34.554749 22546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:34.554785 22542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:34.554826 22541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:42:34.555542 22532 server_base.cc:1034] running on GCE node
I20260808 11:42:34.556103 22532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:42:34.556952 22532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:42:34.558146 22532 hybrid_clock.cc:648] HybridClock initialized: now 1786189354558101 us; error 40 us; skew 500 ppm
I20260808 11:42:34.558425 22532 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:42:34.560459 22532 webserver.cc:460] Webserver started at http://127.24.76.67:35025/ using document root <none> and password file <none>
I20260808 11:42:34.561065 22532 fs_manager.cc:359] Metadata directory not provided
I20260808 11:42:34.561149 22532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:42:34.561398 22532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:42:34.564252 22532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-2/data/instance:
uuid: "9aa08484230d4cc7a50609635951d9da"
format_stamp: "Formatted at 2026-08-08 11:42:34 on dist-test-slave-tc2s"
server_key: "1baae15e6b7ba2f1ffcd6896b13ecd76"
server_key_iv: "4ae0640fdbf52b49702f2c0a6564fccb"
server_key_version: "encryptionkey@0"
I20260808 11:42:34.564962 22532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance:
uuid: "9aa08484230d4cc7a50609635951d9da"
format_stamp: "Formatted at 2026-08-08 11:42:34 on dist-test-slave-tc2s"
server_key: "1baae15e6b7ba2f1ffcd6896b13ecd76"
server_key_iv: "4ae0640fdbf52b49702f2c0a6564fccb"
server_key_version: "encryptionkey@0"
I20260808 11:42:34.569626 22532 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260808 11:42:34.572750 22555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:34.573992 22532 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 11:42:34.574131 22532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-2/wal
uuid: "9aa08484230d4cc7a50609635951d9da"
format_stamp: "Formatted at 2026-08-08 11:42:34 on dist-test-slave-tc2s"
server_key: "1baae15e6b7ba2f1ffcd6896b13ecd76"
server_key_iv: "4ae0640fdbf52b49702f2c0a6564fccb"
server_key_version: "encryptionkey@0"
I20260808 11:42:34.574317 22532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:42:34.588661 22532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:42:34.589350 22532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:42:34.589627 22532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:42:34.590520 22532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:42:34.591850 22532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:42:34.591948 22532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:34.592056 22532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:42:34.592125 22532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:34.621595 22532 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.67:36281
I20260808 11:42:34.621642 22748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.67:36281 every 8 connection(s)
I20260808 11:42:34.622810 22532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
I20260808 11:42:34.632258 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 22532
I20260808 11:42:34.632452 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance
I20260808 11:42:34.633018 24881 external_mini_cluster.cc:1428] Setting key 3180cb74415188dbd5e742bc9b14e75c
I20260808 11:42:34.634595 22316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:34.625231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:33795 (local address 127.24.76.126:34517)
0808 11:42:34.625412 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:34.625419 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:34.626712 (+ 1293us) server_negotiation.cc:407] Connection header received
0808 11:42:34.628418 (+ 1706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:34.628423 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:34.628503 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:34.628592 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:34.629565 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:34.630348 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:34.631909 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:34.632297 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:34.632414 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:34.633769 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:34.633775 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:34.633779 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:42:34.633801 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:34.633838 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:42:34.633853 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:34.633976 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:34.634219 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 11:42:34.634347 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 11:42:34.635545 22751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:34.625703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:34517 (local address 127.24.76.67:33795)
0808 11:42:34.626574 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:34.626609 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:34.628146 (+ 1537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:34.628767 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:34.628775 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:34.628854 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:42:34.629425 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:34.629451 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:34.630506 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:34.630510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:34.631790 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:34.631803 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:34.632527 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:34.632535 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:34.632825 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:34.633499 (+ 674us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:42:34.633502 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:42:34.633538 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:42:34.633613 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:42:34.633996 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:42:34.634007 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:42:34.634016 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 11:42:34.634189 (+ 173us) client_negotiation.cc:240] Negotiation successful
0808 11:42:34.634538 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":295,"threads_started":1}
I20260808 11:42:34.636708 22749 heartbeater.cc:347] Connected to a master server at 127.24.76.126:34517
I20260808 11:42:34.636977 22749 heartbeater.cc:464] Registering TS with master...
I20260808 11:42:34.637669 22749 heartbeater.cc:511] Master 127.24.76.126:34517 requested a full tablet report, sending...
I20260808 11:42:34.638983 21991 ts_manager.cc:194] Registered new tserver with Master: 9aa08484230d4cc7a50609635951d9da (127.24.76.67:36281)
I20260808 11:42:34.639729 21991 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.76.67:33795
I20260808 11:42:34.647737 24881 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:42:34.657029 22316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:34.651671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51012 (local address 127.24.76.126:34517)
0808 11:42:34.651864 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:34.651871 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:34.652274 (+ 403us) server_negotiation.cc:407] Connection header received
0808 11:42:34.652389 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:34.652393 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:34.652462 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:34.652544 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:34.652958 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:34.653618 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:34.654977 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:34.655375 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:34.655461 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:34.656378 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:34.656383 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:34.656386 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:42:34.656404 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:34.656444 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:42:34.656456 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:34.656573 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:34.656693 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 11:42:34.656742 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 11:42:34.660423 21991 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51012:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:42:34.666975 21991 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:42:34.687604 22770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:34.677977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:37363 (local address 127.0.0.1:49710)
0808 11:42:34.678893 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:34.678933 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:34.679146 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:34.680306 (+ 1160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:34.680313 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:34.680341 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:42:34.680766 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:34.680782 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:34.682642 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:34.682648 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:34.684788 (+ 2140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:34.684804 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:34.686645 (+ 1841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:34.686652 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:34.686874 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:34.686924 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 11:42:34.687061 (+ 137us) client_negotiation.cc:240] Negotiation successful
0808 11:42:34.687233 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"mutex_wait_us":25,"thread_start_us":1728,"threads_started":1}
I20260808 11:42:34.687633 22772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:34.678239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49710 (local address 127.24.76.66:37363)
0808 11:42:34.679666 (+ 1427us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:34.679676 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:34.679706 (+ 30us) server_negotiation.cc:407] Connection header received
0808 11:42:34.679828 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:34.679837 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:34.680098 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:34.680296 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:42:34.680955 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:34.682474 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:34.684961 (+ 2487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:34.686413 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:34.686556 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:34.686839 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:34.687103 (+ 264us) server_negotiation.cc:299] Negotiation successful
0808 11:42:34.687270 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1276,"thread_start_us":1164,"threads_started":1}
I20260808 11:42:34.689332 22769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:34.678145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:36281 (local address 127.0.0.1:34516)
0808 11:42:34.679979 (+ 1834us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:34.679996 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:34.680100 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:34.681572 (+ 1472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:34.681577 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:34.681604 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:42:34.682353 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:34.682373 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:34.684239 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:34.684243 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:34.686501 (+ 2258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:34.686516 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:34.688568 (+ 2052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:34.688574 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:34.688747 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:34.688778 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 11:42:34.688871 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 11:42:34.689007 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1704,"mutex_wait_us":3,"thread_start_us":339,"threads_started":1}
I20260808 11:42:34.690390 22773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:34.678591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34516 (local address 127.24.76.67:36281)
0808 11:42:34.681020 (+ 2429us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:34.681029 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:34.681060 (+ 31us) server_negotiation.cc:407] Connection header received
0808 11:42:34.681156 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:34.681162 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:34.681359 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:34.681547 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:42:34.682614 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:34.684102 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:34.686736 (+ 2634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:34.688048 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:34.688155 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:34.688400 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:34.689917 (+ 1517us) server_negotiation.cc:299] Negotiation successful
0808 11:42:34.690083 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2249,"thread_start_us":2099,"threads_started":1}
I20260808 11:42:34.690445 22775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:34.679258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:38039 (local address 127.0.0.1:49684)
0808 11:42:34.682556 (+ 3298us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:34.682573 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:34.682681 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:34.683453 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:34.683457 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:34.683482 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:42:34.683891 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:34.683902 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:34.685574 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:34.685580 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:34.687673 (+ 2093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:34.687688 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:34.689502 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:34.689508 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:34.689687 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:34.689912 (+ 225us) client_negotiation.cc:769] Sending connection context
0808 11:42:34.690003 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 11:42:34.690122 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3180,"mutex_wait_us":7,"thread_start_us":3037,"threads_started":1}
I20260808 11:42:34.690544 22778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:34.679742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49684 (local address 127.24.76.65:38039)
0808 11:42:34.682148 (+ 2406us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:34.682156 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:34.682811 (+ 655us) server_negotiation.cc:407] Connection header received
0808 11:42:34.682911 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:34.682917 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:34.683141 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:34.683316 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:42:34.684092 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:34.685403 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:34.688104 (+ 2701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:34.689037 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:34.689161 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:34.689378 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:34.690039 (+ 661us) server_negotiation.cc:299] Negotiation successful
0808 11:42:34.690197 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2248,"thread_start_us":2153,"threads_started":1}
I20260808 11:42:34.691715 22422 tablet_service.cc:1467] Processing CreateTablet for tablet ba78922633bd4fdf8759ac353bdf6f2d (DEFAULT_TABLE table=test-table [id=2aec56bda4174d9b9fb6dccfd8e74707]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:42:34.693250 22422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba78922633bd4fdf8759ac353bdf6f2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:42:34.693637 22644 tablet_service.cc:1467] Processing CreateTablet for tablet ba78922633bd4fdf8759ac353bdf6f2d (DEFAULT_TABLE table=test-table [id=2aec56bda4174d9b9fb6dccfd8e74707]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:42:34.693958 22197 tablet_service.cc:1467] Processing CreateTablet for tablet ba78922633bd4fdf8759ac353bdf6f2d (DEFAULT_TABLE table=test-table [id=2aec56bda4174d9b9fb6dccfd8e74707]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:42:34.694942 22644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba78922633bd4fdf8759ac353bdf6f2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:42:34.695231 22197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba78922633bd4fdf8759ac353bdf6f2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:42:34.705952 22779 tablet_bootstrap.cc:492] T ba78922633bd4fdf8759ac353bdf6f2d P 9aa08484230d4cc7a50609635951d9da: Bootstrap starting.
I20260808 11:42:34.707792 22780 tablet_bootstrap.cc:492] T ba78922633bd4fdf8759ac353bdf6f2d P 8f5fe547fc8747c586c2637d19f6b947: Bootstrap starting.
I20260808 11:42:34.709131 22782 tablet_bootstrap.cc:492] T ba78922633bd4fdf8759ac353bdf6f2d P b41557f51bbf4627ae4efaa6f40cbe28: Bootstrap starting.
I20260808 11:42:34.709569 22779 tablet_bootstrap.cc:654] T ba78922633bd4fdf8759ac353bdf6f2d P 9aa08484230d4cc7a50609635951d9da: Neither blocks nor log segments found. Creating new log.
I20260808 11:42:34.710886 22779 log.cc:824] T ba78922633bd4fdf8759ac353bdf6f2d P 9aa08484230d4cc7a50609635951d9da: Log is configured to *not* fsync() on all Append() calls
I20260808 11:42:34.711022 22780 tablet_bootstrap.cc:654] T ba78922633bd4fdf8759ac353bdf6f2d P 8f5fe547fc8747c586c2637d19f6b947: Neither blocks nor log segments found. Creating new log.
I20260808 11:42:34.712077 22782 tablet_bootstrap.cc:654] T ba78922633bd4fdf8759ac353bdf6f2d P b41557f51bbf4627ae4efaa6f40cbe28: Neither blocks nor log segments found. Creating new log.
I20260808 11:42:34.712236 22780 log.cc:824] T ba78922633bd4fdf8759ac353bdf6f2d P 8f5fe547fc8747c586c2637d19f6b947: Log is configured to *not* fsync() on all Append() calls
I20260808 11:42:34.713297 22782 log.cc:824] T ba78922633bd4fdf8759ac353bdf6f2d P b41557f51bbf4627ae4efaa6f40cbe28: Log is configured to *not* fsync() on all Append() calls
I20260808 11:42:34.715139 22779 tablet_bootstrap.cc:492] T ba78922633bd4fdf8759ac353bdf6f2d P 9aa08484230d4cc7a50609635951d9da: No bootstrap required, opened a new log
I20260808 11:42:34.715281 22780 tablet_bootstrap.cc:492] T ba78922633bd4fdf8759ac353bdf6f2d P 8f5fe547fc8747c586c2637d19f6b947: No bootstrap required, opened a new log
I20260808 11:42:34.715391 22779 ts_tablet_manager.cc:1397] T ba78922633bd4fdf8759ac353bdf6f2d P 9aa08484230d4cc7a50609635951d9da: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.007s
I20260808 11:42:34.715551 22780 ts_tablet_manager.cc:1397] T ba78922633bd4fdf8759ac353bdf6f2d P 8f5fe547fc8747c586c2637d19f6b947: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20260808 11:42:34.716184 22782 tablet_bootstrap.cc:492] T ba78922633bd4fdf8759ac353bdf6f2d P b41557f51bbf4627ae4efaa6f40cbe28: No bootstrap required, opened a new log
I20260808 11:42:34.716439 22782 ts_tablet_manager.cc:1397] T ba78922633bd4fdf8759ac353bdf6f2d P b41557f51bbf4627ae4efaa6f40cbe28: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260808 11:42:34.722575 22779 raft_consensus.cc:348] T ba78922633bd4fdf8759ac353bdf6f2d P 9aa08484230d4cc7a50609635951d9da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f5fe547fc8747c586c2637d19f6b947" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37363 } } peers { permanent_uuid: "9aa08484230d4cc7a50609635951d9da" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 36281 } } peers { permanent_uuid: "b41557f51bbf4627ae4efaa6f40cbe28" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 38039 } }
I20260808 11:42:34.722923 22779 raft_consensus.cc:374] T ba78922633bd4fdf8759ac353bdf6f2d P 9aa08484230d4cc7a50609635951d9da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:42:34.723030 22779 raft_consensus.cc:729] T ba78922633bd4fdf8759ac353bdf6f2d P 9aa08484230d4cc7a50609635951d9da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa08484230d4cc7a50609635951d9da, State: Initialized, Role: FOLLOWER
I20260808 11:42:34.723557 22779 consensus_queue.cc:260] T ba78922633bd4fdf8759ac353bdf6f2d P 9aa08484230d4cc7a50609635951d9da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f5fe547fc8747c586c2637d19f6b947" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37363 } } peers { permanent_uuid: "9aa08484230d4cc7a50609635951d9da" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 36281 } } peers { permanent_uuid: "b41557f51bbf4627ae4efaa6f40cbe28" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 38039 } }
I20260808 11:42:34.724579 22749 heartbeater.cc:503] Master 127.24.76.126:34517 was elected leader, sending a full tablet report...
I20260808 11:42:34.725147 22779 ts_tablet_manager.cc:1428] T ba78922633bd4fdf8759ac353bdf6f2d P 9aa08484230d4cc7a50609635951d9da: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260808 11:42:34.725517 22780 raft_consensus.cc:348] T ba78922633bd4fdf8759ac353bdf6f2d P 8f5fe547fc8747c586c2637d19f6b947 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f5fe547fc8747c586c2637d19f6b947" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37363 } } peers { permanent_uuid: "9aa08484230d4cc7a50609635951d9da" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 36281 } } peers { permanent_uuid: "b41557f51bbf4627ae4efaa6f40cbe28" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 38039 } }
I20260808 11:42:34.725932 22780 raft_consensus.cc:374] T ba78922633bd4fdf8759ac353bdf6f2d P 8f5fe547fc8747c586c2637d19f6b947 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:42:34.726081 22780 raft_consensus.cc:729] T ba78922633bd4fdf8759ac353bdf6f2d P 8f5fe547fc8747c586c2637d19f6b947 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f5fe547fc8747c586c2637d19f6b947, State: Initialized, Role: FOLLOWER
I20260808 11:42:34.726601 22780 consensus_queue.cc:260] T ba78922633bd4fdf8759ac353bdf6f2d P 8f5fe547fc8747c586c2637d19f6b947 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f5fe547fc8747c586c2637d19f6b947" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37363 } } peers { permanent_uuid: "9aa08484230d4cc7a50609635951d9da" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 36281 } } peers { permanent_uuid: "b41557f51bbf4627ae4efaa6f40cbe28" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 38039 } }
I20260808 11:42:34.726768 22782 raft_consensus.cc:348] T ba78922633bd4fdf8759ac353bdf6f2d P b41557f51bbf4627ae4efaa6f40cbe28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f5fe547fc8747c586c2637d19f6b947" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37363 } } peers { permanent_uuid: "9aa08484230d4cc7a50609635951d9da" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 36281 } } peers { permanent_uuid: "b41557f51bbf4627ae4efaa6f40cbe28" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 38039 } }
I20260808 11:42:34.727205 22782 raft_consensus.cc:374] T ba78922633bd4fdf8759ac353bdf6f2d P b41557f51bbf4627ae4efaa6f40cbe28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:42:34.727340 22782 raft_consensus.cc:729] T ba78922633bd4fdf8759ac353bdf6f2d P b41557f51bbf4627ae4efaa6f40cbe28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b41557f51bbf4627ae4efaa6f40cbe28, State: Initialized, Role: FOLLOWER
I20260808 11:42:34.727639 22529 heartbeater.cc:503] Master 127.24.76.126:34517 was elected leader, sending a full tablet report...
I20260808 11:42:34.727905 22782 consensus_queue.cc:260] T ba78922633bd4fdf8759ac353bdf6f2d P b41557f51bbf4627ae4efaa6f40cbe28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f5fe547fc8747c586c2637d19f6b947" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37363 } } peers { permanent_uuid: "9aa08484230d4cc7a50609635951d9da" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 36281 } } peers { permanent_uuid: "b41557f51bbf4627ae4efaa6f40cbe28" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 38039 } }
I20260808 11:42:34.728148 22780 ts_tablet_manager.cc:1428] T ba78922633bd4fdf8759ac353bdf6f2d P 8f5fe547fc8747c586c2637d19f6b947: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260808 11:42:34.729303 22313 heartbeater.cc:503] Master 127.24.76.126:34517 was elected leader, sending a full tablet report...
I20260808 11:42:34.729866 22782 ts_tablet_manager.cc:1428] T ba78922633bd4fdf8759ac353bdf6f2d P b41557f51bbf4627ae4efaa6f40cbe28: Time spent starting tablet: real 0.013s user 0.011s sys 0.003s
W20260808 11:42:34.736200 22315 tablet.cc:2369] T ba78922633bd4fdf8759ac353bdf6f2d P b41557f51bbf4627ae4efaa6f40cbe28: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:42:34.842002 22790 raft_consensus.cc:482] T ba78922633bd4fdf8759ac353bdf6f2d P 8f5fe547fc8747c586c2637d19f6b947 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:42:34.842351 22790 raft_consensus.cc:504] T ba78922633bd4fdf8759ac353bdf6f2d P 8f5fe547fc8747c586c2637d19f6b947 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f5fe547fc8747c586c2637d19f6b947" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37363 } } peers { permanent_uuid: "9aa08484230d4cc7a50609635951d9da" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 36281 } } peers { permanent_uuid: "b41557f51bbf4627ae4efaa6f40cbe28" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 38039 } }
I20260808 11:42:34.844146 22790 leader_election.cc:288] T ba78922633bd4fdf8759ac353bdf6f2d P 8f5fe547fc8747c586c2637d19f6b947 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa08484230d4cc7a50609635951d9da (127.24.76.67:36281), b41557f51bbf4627ae4efaa6f40cbe28 (127.24.76.65:38039)
I20260808 11:42:34.851825 22773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:34.844929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:39519 (local address 127.24.76.67:36281)
0808 11:42:34.845183 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:34.845189 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:34.845321 (+ 132us) server_negotiation.cc:407] Connection header received
0808 11:42:34.845567 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:34.845574 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:34.845661 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:34.845759 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:42:34.846412 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:34.847368 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:34.849730 (+ 2362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:34.850901 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:34.851071 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:34.851142 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:34.851500 (+ 358us) server_negotiation.cc:299] Negotiation successful
0808 11:42:34.851600 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260808 11:42:34.851927 22795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:34.844498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:36281 (local address 127.24.76.66:39519)
0808 11:42:34.845228 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:34.845245 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:34.845386 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:34.845789 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:34.845794 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:34.845826 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:42:34.846278 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:34.846289 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:34.847490 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:34.847494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:34.849550 (+ 2056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:34.849564 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:34.851118 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:34.851123 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:34.851264 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:34.851342 (+ 78us) client_negotiation.cc:769] Sending connection context
0808 11:42:34.851448 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 11:42:34.851584 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"mutex_wait_us":2,"thread_start_us":384,"threads_started":1}
I20260808 11:42:34.852118 22778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:34.845501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:36757 (local address 127.24.76.65:38039)
0808 11:42:34.845677 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:34.845682 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:34.846151 (+ 469us) server_negotiation.cc:407] Connection header received
0808 11:42:34.846314 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:34.846317 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:34.846399 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:34.846510 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:42:34.846944 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:34.847802 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:34.850107 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:34.851248 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:34.851363 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:34.851407 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:34.851764 (+ 357us) server_negotiation.cc:299] Negotiation successful
0808 11:42:34.851884 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 11:42:34.852206 22796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:34.845307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:38039 (local address 127.24.76.66:36757)
0808 11:42:34.846052 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:34.846067 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:34.846199 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:34.846515 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:34.846518 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:34.846537 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:42:34.846801 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:34.846816 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:34.847921 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:34.847926 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:34.849926 (+ 2000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:34.849940 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:34.851432 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:34.851439 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:34.851613 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:34.851641 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 11:42:34.851725 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 11:42:34.851846 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"mutex_wait_us":2,"thread_start_us":390,"threads_started":1}
I20260808 11:42:34.853104 22676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba78922633bd4fdf8759ac353bdf6f2d" candidate_uuid: "8f5fe547fc8747c586c2637d19f6b947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa08484230d4cc7a50609635951d9da" is_pre_election: true
I20260808 11:42:34.853215 22233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba78922633bd4fdf8759ac353bdf6f2d" candidate_uuid: "8f5fe547fc8747c586c2637d19f6b947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b41557f51bbf4627ae4efaa6f40cbe28" is_pre_election: true
I20260808 11:42:34.853605 22676 raft_consensus.cc:2454] T ba78922633bd4fdf8759ac353bdf6f2d P 9aa08484230d4cc7a50609635951d9da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f5fe547fc8747c586c2637d19f6b947 in term 0.
I20260808 11:42:34.853717 22233 raft_consensus.cc:2454] T ba78922633bd4fdf8759ac353bdf6f2d P b41557f51bbf4627ae4efaa6f40cbe28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f5fe547fc8747c586c2637d19f6b947 in term 0.
I20260808 11:42:34.854223 22339 leader_election.cc:302] T ba78922633bd4fdf8759ac353bdf6f2d P 8f5fe547fc8747c586c2637d19f6b947 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f5fe547fc8747c586c2637d19f6b947, 9aa08484230d4cc7a50609635951d9da; no voters:
I20260808 11:42:34.854580 22790 raft_consensus.cc:2781] T ba78922633bd4fdf8759ac353bdf6f2d P 8f5fe547fc8747c586c2637d19f6b947 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:42:34.854733 22790 raft_consensus.cc:482] T ba78922633bd4fdf8759ac353bdf6f2d P 8f5fe547fc8747c586c2637d19f6b947 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:42:34.854884 22790 raft_consensus.cc:3037] T ba78922633bd4fdf8759ac353bdf6f2d P 8f5fe547fc8747c586c2637d19f6b947 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:42:34.857043 22790 raft_consensus.cc:504] T ba78922633bd4fdf8759ac353bdf6f2d P 8f5fe547fc8747c586c2637d19f6b947 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f5fe547fc8747c586c2637d19f6b947" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37363 } } peers { permanent_uuid: "9aa08484230d4cc7a50609635951d9da" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 36281 } } peers { permanent_uuid: "b41557f51bbf4627ae4efaa6f40cbe28" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 38039 } }
I20260808 11:42:34.857645 22790 leader_election.cc:288] T ba78922633bd4fdf8759ac353bdf6f2d P 8f5fe547fc8747c586c2637d19f6b947 [CANDIDATE]: Term 1 election: Requested vote from peers 9aa08484230d4cc7a50609635951d9da (127.24.76.67:36281), b41557f51bbf4627ae4efaa6f40cbe28 (127.24.76.65:38039)
I20260808 11:42:34.858152 22233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba78922633bd4fdf8759ac353bdf6f2d" candidate_uuid: "8f5fe547fc8747c586c2637d19f6b947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b41557f51bbf4627ae4efaa6f40cbe28"
I20260808 11:42:34.858201 22676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba78922633bd4fdf8759ac353bdf6f2d" candidate_uuid: "8f5fe547fc8747c586c2637d19f6b947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa08484230d4cc7a50609635951d9da"
I20260808 11:42:34.858328 22233 raft_consensus.cc:3037] T ba78922633bd4fdf8759ac353bdf6f2d P b41557f51bbf4627ae4efaa6f40cbe28 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:42:34.858385 22676 raft_consensus.cc:3037] T ba78922633bd4fdf8759ac353bdf6f2d P 9aa08484230d4cc7a50609635951d9da [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:42:34.860526 22233 raft_consensus.cc:2454] T ba78922633bd4fdf8759ac353bdf6f2d P b41557f51bbf4627ae4efaa6f40cbe28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f5fe547fc8747c586c2637d19f6b947 in term 1.
I20260808 11:42:34.860805 22676 raft_consensus.cc:2454] T ba78922633bd4fdf8759ac353bdf6f2d P 9aa08484230d4cc7a50609635951d9da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f5fe547fc8747c586c2637d19f6b947 in term 1.
I20260808 11:42:34.860895 22339 leader_election.cc:302] T ba78922633bd4fdf8759ac353bdf6f2d P 8f5fe547fc8747c586c2637d19f6b947 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f5fe547fc8747c586c2637d19f6b947, b41557f51bbf4627ae4efaa6f40cbe28; no voters:
I20260808 11:42:34.861155 22790 raft_consensus.cc:2781] T ba78922633bd4fdf8759ac353bdf6f2d P 8f5fe547fc8747c586c2637d19f6b947 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:42:34.861562 22790 raft_consensus.cc:686] T ba78922633bd4fdf8759ac353bdf6f2d P 8f5fe547fc8747c586c2637d19f6b947 [term 1 LEADER]: Becoming Leader. State: Replica: 8f5fe547fc8747c586c2637d19f6b947, State: Running, Role: LEADER
I20260808 11:42:34.862061 22790 consensus_queue.cc:237] T ba78922633bd4fdf8759ac353bdf6f2d P 8f5fe547fc8747c586c2637d19f6b947 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f5fe547fc8747c586c2637d19f6b947" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37363 } } peers { permanent_uuid: "9aa08484230d4cc7a50609635951d9da" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 36281 } } peers { permanent_uuid: "b41557f51bbf4627ae4efaa6f40cbe28" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 38039 } }
I20260808 11:42:34.866423 21987 catalog_manager.cc:5495] T ba78922633bd4fdf8759ac353bdf6f2d P 8f5fe547fc8747c586c2637d19f6b947 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f5fe547fc8747c586c2637d19f6b947 (127.24.76.66). New cstate: current_term: 1 leader_uuid: "8f5fe547fc8747c586c2637d19f6b947" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f5fe547fc8747c586c2637d19f6b947" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 37363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa08484230d4cc7a50609635951d9da" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 36281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b41557f51bbf4627ae4efaa6f40cbe28" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 38039 } health_report { overall_health: UNKNOWN } } }
W20260808 11:42:34.874365 22750 tablet.cc:2369] T ba78922633bd4fdf8759ac353bdf6f2d P 9aa08484230d4cc7a50609635951d9da: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:42:34.882136 22772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:34.877194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49726 (local address 127.24.76.66:37363)
0808 11:42:34.877379 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:34.877385 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:34.877405 (+ 20us) server_negotiation.cc:407] Connection header received
0808 11:42:34.877521 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:34.877526 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:34.877632 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:34.877750 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:42:34.878172 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:34.878911 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:34.880240 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:34.880636 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:34.880720 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:34.881142 (+ 422us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:42:34.881596 (+ 454us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:42:34.881719 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:34.881801 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 11:42:34.881836 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 11:42:34.889706 22233 raft_consensus.cc:1261] T ba78922633bd4fdf8759ac353bdf6f2d P b41557f51bbf4627ae4efaa6f40cbe28 [term 1 FOLLOWER]: Refusing update from remote peer 8f5fe547fc8747c586c2637d19f6b947: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:42:34.889773 22676 raft_consensus.cc:1261] T ba78922633bd4fdf8759ac353bdf6f2d P 9aa08484230d4cc7a50609635951d9da [term 1 FOLLOWER]: Refusing update from remote peer 8f5fe547fc8747c586c2637d19f6b947: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:42:34.890584 22797 consensus_queue.cc:1035] T ba78922633bd4fdf8759ac353bdf6f2d P 8f5fe547fc8747c586c2637d19f6b947 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b41557f51bbf4627ae4efaa6f40cbe28" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 38039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:42:34.890905 22790 consensus_queue.cc:1035] T ba78922633bd4fdf8759ac353bdf6f2d P 8f5fe547fc8747c586c2637d19f6b947 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa08484230d4cc7a50609635951d9da" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 36281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:42:34.899739 22800 mvcc.cc:205] Tried to move back new op lower bound from 7316231597619568640 to 7316231597518651392. Current Snapshot: MvccSnapshot[applied={T|T < 7316231597619568640}]
I20260808 11:42:34.906925 22804 mvcc.cc:205] Tried to move back new op lower bound from 7316231597619568640 to 7316231597518651392. Current Snapshot: MvccSnapshot[applied={T|T < 7316231597619568640}]
I20260808 11:42:34.916610 21989 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51012:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:42:34.916934 21989 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51012:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:42:34.920713 21989 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0b081a89d3e647258a8bbdb7b78dab8d: Sending DeleteTablet for 3 replicas of tablet ba78922633bd4fdf8759ac353bdf6f2d
I20260808 11:42:34.921725 22422 tablet_service.cc:1514] Processing DeleteTablet for tablet ba78922633bd4fdf8759ac353bdf6f2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:42:34 UTC) from {username='slave'} at 127.0.0.1:49710
I20260808 11:42:34.921851 22197 tablet_service.cc:1514] Processing DeleteTablet for tablet ba78922633bd4fdf8759ac353bdf6f2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:42:34 UTC) from {username='slave'} at 127.0.0.1:49684
I20260808 11:42:34.921860 22644 tablet_service.cc:1514] Processing DeleteTablet for tablet ba78922633bd4fdf8759ac353bdf6f2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:42:34 UTC) from {username='slave'} at 127.0.0.1:34516
I20260808 11:42:34.922916 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 22095
I20260808 11:42:34.922971 22816 tablet_replica.cc:330] T ba78922633bd4fdf8759ac353bdf6f2d P 8f5fe547fc8747c586c2637d19f6b947: stopping tablet replica
I20260808 11:42:34.923367 22816 raft_consensus.cc:2229] T ba78922633bd4fdf8759ac353bdf6f2d P 8f5fe547fc8747c586c2637d19f6b947 [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:42:34.924024 22816 raft_consensus.cc:2258] T ba78922633bd4fdf8759ac353bdf6f2d P 8f5fe547fc8747c586c2637d19f6b947 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:42:34.926597 22816 ts_tablet_manager.cc:1905] T ba78922633bd4fdf8759ac353bdf6f2d P 8f5fe547fc8747c586c2637d19f6b947: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:42:34.930029 22817 tablet_replica.cc:330] T ba78922633bd4fdf8759ac353bdf6f2d P b41557f51bbf4627ae4efaa6f40cbe28: stopping tablet replica
I20260808 11:42:34.930874 22818 tablet_replica.cc:330] T ba78922633bd4fdf8759ac353bdf6f2d P 9aa08484230d4cc7a50609635951d9da: stopping tablet replica
I20260808 11:42:34.931259 22818 raft_consensus.cc:2229] T ba78922633bd4fdf8759ac353bdf6f2d P 9aa08484230d4cc7a50609635951d9da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:42:34.930415 22817 raft_consensus.cc:2229] T ba78922633bd4fdf8759ac353bdf6f2d P b41557f51bbf4627ae4efaa6f40cbe28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:42:34.932184 22817 raft_consensus.cc:2258] T ba78922633bd4fdf8759ac353bdf6f2d P b41557f51bbf4627ae4efaa6f40cbe28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:42:34.932602 22818 raft_consensus.cc:2258] T ba78922633bd4fdf8759ac353bdf6f2d P 9aa08484230d4cc7a50609635951d9da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:42:34.934687 22816 ts_tablet_manager.cc:1918] T ba78922633bd4fdf8759ac353bdf6f2d P 8f5fe547fc8747c586c2637d19f6b947: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:42:34.934882 22817 ts_tablet_manager.cc:1905] T ba78922633bd4fdf8759ac353bdf6f2d P b41557f51bbf4627ae4efaa6f40cbe28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:42:34.934926 22816 log.cc:1196] T ba78922633bd4fdf8759ac353bdf6f2d P 8f5fe547fc8747c586c2637d19f6b947: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-1/wal/wals/ba78922633bd4fdf8759ac353bdf6f2d
I20260808 11:42:34.935556 22816 ts_tablet_manager.cc:1939] T ba78922633bd4fdf8759ac353bdf6f2d P 8f5fe547fc8747c586c2637d19f6b947: Deleting consensus metadata
I20260808 11:42:34.936240 22818 ts_tablet_manager.cc:1905] T ba78922633bd4fdf8759ac353bdf6f2d P 9aa08484230d4cc7a50609635951d9da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:42:34.937517 21959 catalog_manager.cc:4841] TS 8f5fe547fc8747c586c2637d19f6b947 (127.24.76.66:37363): tablet ba78922633bd4fdf8759ac353bdf6f2d (table test-table [id=2aec56bda4174d9b9fb6dccfd8e74707]) successfully deleted
I20260808 11:42:34.942137 22817 ts_tablet_manager.cc:1918] T ba78922633bd4fdf8759ac353bdf6f2d P b41557f51bbf4627ae4efaa6f40cbe28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:42:34.942368 22817 log.cc:1196] T ba78922633bd4fdf8759ac353bdf6f2d P b41557f51bbf4627ae4efaa6f40cbe28: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-0/wal/wals/ba78922633bd4fdf8759ac353bdf6f2d
I20260808 11:42:34.942965 22817 ts_tablet_manager.cc:1939] T ba78922633bd4fdf8759ac353bdf6f2d P b41557f51bbf4627ae4efaa6f40cbe28: Deleting consensus metadata
I20260808 11:42:34.944640 22818 ts_tablet_manager.cc:1918] T ba78922633bd4fdf8759ac353bdf6f2d P 9aa08484230d4cc7a50609635951d9da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:42:34.944861 22818 log.cc:1196] T ba78922633bd4fdf8759ac353bdf6f2d P 9aa08484230d4cc7a50609635951d9da: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786189042529715-24881-0/minicluster-data/ts-2/wal/wals/ba78922633bd4fdf8759ac353bdf6f2d
I20260808 11:42:34.945134 21956 catalog_manager.cc:4841] TS b41557f51bbf4627ae4efaa6f40cbe28 (127.24.76.65:38039): tablet ba78922633bd4fdf8759ac353bdf6f2d (table test-table [id=2aec56bda4174d9b9fb6dccfd8e74707]) successfully deleted
I20260808 11:42:34.945452 22818 ts_tablet_manager.cc:1939] T ba78922633bd4fdf8759ac353bdf6f2d P 9aa08484230d4cc7a50609635951d9da: Deleting consensus metadata
I20260808 11:42:34.946656 22778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:34.924365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49700 (local address 127.24.76.65:38039)
0808 11:42:34.924554 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:34.924561 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:34.938454 (+ 13893us) server_negotiation.cc:407] Connection header received
0808 11:42:34.938769 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:34.938774 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:34.938897 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:34.938969 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:34.939620 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:34.940814 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:34.942878 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:34.943543 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:34.943647 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:34.945075 (+ 1428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:34.945086 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:34.945107 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:42:34.945829 (+ 722us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:34.945887 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:42:34.945938 (+ 51us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:34.946060 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:34.946198 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 11:42:34.946306 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 11:42:34.947280 21956 catalog_manager.cc:4841] TS 9aa08484230d4cc7a50609635951d9da (127.24.76.67:36281): tablet ba78922633bd4fdf8759ac353bdf6f2d (table test-table [id=2aec56bda4174d9b9fb6dccfd8e74707]) successfully deleted
I20260808 11:42:34.947357 22304 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:42:35.043503 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 22095
I20260808 11:42:35.064309 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 22318
I20260808 11:42:35.071921 22772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:35.065096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49728 (local address 127.24.76.66:37363)
0808 11:42:35.065297 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:35.065303 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:35.065325 (+ 22us) server_negotiation.cc:407] Connection header received
0808 11:42:35.065452 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:35.065458 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:35.065582 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:35.065712 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:35.066140 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:35.067329 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:35.068578 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:35.069208 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:35.069347 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:35.070380 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:35.070393 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:35.070415 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:42:35.071189 (+ 774us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:35.071250 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:42:35.071277 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:35.071407 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:35.071470 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 11:42:35.071563 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 11:42:35.072624 22523 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:42:35.183458 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 22318
I20260808 11:42:35.209185 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 22532
I20260808 11:42:35.215970 22773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:35.210035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34520 (local address 127.24.76.67:36281)
0808 11:42:35.210215 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:35.210220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:35.210240 (+ 20us) server_negotiation.cc:407] Connection header received
0808 11:42:35.210363 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:35.210368 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:35.210463 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:35.210562 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:35.210991 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:35.211755 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:35.213062 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:35.213457 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:35.213575 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:35.214679 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:35.214693 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:35.214714 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:42:35.215263 (+ 549us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:35.215312 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:42:35.215333 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:35.215445 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:35.215573 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 11:42:35.215685 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 11:42:35.216598 22739 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:42:35.304934 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 22532
I20260808 11:42:35.326876 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 21932
I20260808 11:42:35.327991 22047 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:42:35.437438 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 21932
2026-08-08T11:42:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2757 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:42:35 dist-test-slave-tc2s krb5kdc[22829](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:42:35 dist-test-slave-tc2s krb5kdc[22829](info): set up 2 sockets
Aug 08 11:42:35 dist-test-slave-tc2s krb5kdc[22829](info): commencing operation
krb5kdc: starting...
W20260808 11:42:36.661139 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.011s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:42:36 dist-test-slave-tc2s krb5kdc[22829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:42:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:42:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
I20260808 11:42:36.954571 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:39183
--webserver_interface=127.24.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:33009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:39183
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:42:37.298028 22850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:42:37.298405 22850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:42:37.298549 22850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:42:37.305835 22850 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:42:37.305961 22850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:42:37.306043 22850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:42:37.306121 22850 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:42:37.306185 22850 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:42:37.315689 22850 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:33009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:39183
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:39183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:42:37.316982 22850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:42:37.318454 22850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:42:37.329881 22860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:37.330003 22861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:37.330204 22863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:42:37.330843 22850 server_base.cc:1034] running on GCE node
I20260808 11:42:37.331559 22850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:42:37.332862 22850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:42:37.334070 22850 hybrid_clock.cc:648] HybridClock initialized: now 1786189357334022 us; error 30 us; skew 500 ppm
I20260808 11:42:37.334393 22850 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:42:37 dist-test-slave-tc2s krb5kdc[22829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189357, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:42:37.337281 22850 init.cc:377] Logged in from keytab as kudu/127.24.76.126@KRBTEST.COM (short username kudu)
I20260808 11:42:37.338721 22850 webserver.cc:460] Webserver started at http://127.24.76.126:39361/ using document root <none> and password file <none>
I20260808 11:42:37.339354 22850 fs_manager.cc:359] Metadata directory not provided
I20260808 11:42:37.339468 22850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:42:37.339757 22850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:42:37.342679 22850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "f14d4e75fbf04cdb99b3c2ca4fb7f556"
format_stamp: "Formatted at 2026-08-08 11:42:37 on dist-test-slave-tc2s"
server_key: "1edc802b9373923934dc6a2d60a3de48"
server_key_iv: "9c39b137b47c05c4cc16252840f18c0c"
server_key_version: "encryptionkey@0"
I20260808 11:42:37.343382 22850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "f14d4e75fbf04cdb99b3c2ca4fb7f556"
format_stamp: "Formatted at 2026-08-08 11:42:37 on dist-test-slave-tc2s"
server_key: "1edc802b9373923934dc6a2d60a3de48"
server_key_iv: "9c39b137b47c05c4cc16252840f18c0c"
server_key_version: "encryptionkey@0"
I20260808 11:42:37.348277 22850 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 11:42:37.351575 22872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:37.352746 22850 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 11:42:37.352892 22850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "f14d4e75fbf04cdb99b3c2ca4fb7f556"
format_stamp: "Formatted at 2026-08-08 11:42:37 on dist-test-slave-tc2s"
server_key: "1edc802b9373923934dc6a2d60a3de48"
server_key_iv: "9c39b137b47c05c4cc16252840f18c0c"
server_key_version: "encryptionkey@0"
I20260808 11:42:37.353070 22850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:42:37.382133 22850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:42:37.386004 22850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:42:37.386264 22850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:42:37.403865 22850 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:39183
I20260808 11:42:37.403895 22953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:39183 every 8 connection(s)
I20260808 11:42:37.405575 22850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:42:37.409996 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 22850
I20260808 11:42:37.410071 22954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:42:37.410192 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:42:37.410763 24881 external_mini_cluster.cc:1428] Setting key 34f6aa01b959b8131ef640074a89f462
I20260808 11:42:37.423570 22954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f14d4e75fbf04cdb99b3c2ca4fb7f556: Bootstrap starting.
Aug 08 11:42:37 dist-test-slave-tc2s krb5kdc[22829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:42:37.427335 22954 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f14d4e75fbf04cdb99b3c2ca4fb7f556: Neither blocks nor log segments found. Creating new log.
I20260808 11:42:37.428479 22954 log.cc:824] T 00000000000000000000000000000000 P f14d4e75fbf04cdb99b3c2ca4fb7f556: Log is configured to *not* fsync() on all Append() calls
I20260808 11:42:37.431529 22954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f14d4e75fbf04cdb99b3c2ca4fb7f556: No bootstrap required, opened a new log
I20260808 11:42:37.434657 22958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:37.413076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40562 (local address 127.24.76.126:39183)
0808 11:42:37.413841 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:37.413852 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:37.413923 (+ 71us) server_negotiation.cc:407] Connection header received
0808 11:42:37.415964 (+ 2041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:37.415988 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:37.416355 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:37.416672 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:37.418555 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:37.419895 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:37.421707 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:37.422384 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:37.422524 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:37.427388 (+ 4864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:37.427396 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:37.427409 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:42:37.427453 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:37.430672 (+ 3219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:37.431446 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:37.431451 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:37.431457 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:37.431628 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:37.432052 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:37.432056 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:37.432058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:37.432556 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:37.432785 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:37.433262 (+ 477us) server_negotiation.cc:299] Negotiation successful
0808 11:42:37.433572 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":351,"threads_started":1}
I20260808 11:42:37.439371 22954 raft_consensus.cc:348] T 00000000000000000000000000000000 P f14d4e75fbf04cdb99b3c2ca4fb7f556 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f14d4e75fbf04cdb99b3c2ca4fb7f556" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39183 } }
I20260808 11:42:37.439719 22954 raft_consensus.cc:374] T 00000000000000000000000000000000 P f14d4e75fbf04cdb99b3c2ca4fb7f556 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:42:37.439817 22954 raft_consensus.cc:729] T 00000000000000000000000000000000 P f14d4e75fbf04cdb99b3c2ca4fb7f556 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f14d4e75fbf04cdb99b3c2ca4fb7f556, State: Initialized, Role: FOLLOWER
I20260808 11:42:37.440326 22954 consensus_queue.cc:260] T 00000000000000000000000000000000 P f14d4e75fbf04cdb99b3c2ca4fb7f556 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f14d4e75fbf04cdb99b3c2ca4fb7f556" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39183 } }
I20260808 11:42:37.440557 22954 raft_consensus.cc:388] T 00000000000000000000000000000000 P f14d4e75fbf04cdb99b3c2ca4fb7f556 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:42:37.440680 22954 raft_consensus.cc:482] T 00000000000000000000000000000000 P f14d4e75fbf04cdb99b3c2ca4fb7f556 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:42:37.440819 22954 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f14d4e75fbf04cdb99b3c2ca4fb7f556 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:42:37.442909 22954 raft_consensus.cc:504] T 00000000000000000000000000000000 P f14d4e75fbf04cdb99b3c2ca4fb7f556 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f14d4e75fbf04cdb99b3c2ca4fb7f556" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39183 } }
I20260808 11:42:37.443368 22954 leader_election.cc:302] T 00000000000000000000000000000000 P f14d4e75fbf04cdb99b3c2ca4fb7f556 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f14d4e75fbf04cdb99b3c2ca4fb7f556; no voters:
I20260808 11:42:37.443984 22954 leader_election.cc:288] T 00000000000000000000000000000000 P f14d4e75fbf04cdb99b3c2ca4fb7f556 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:42:37.444087 22961 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f14d4e75fbf04cdb99b3c2ca4fb7f556 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:42:37.444763 22961 raft_consensus.cc:686] T 00000000000000000000000000000000 P f14d4e75fbf04cdb99b3c2ca4fb7f556 [term 1 LEADER]: Becoming Leader. State: Replica: f14d4e75fbf04cdb99b3c2ca4fb7f556, State: Running, Role: LEADER
I20260808 11:42:37.445269 22961 consensus_queue.cc:237] T 00000000000000000000000000000000 P f14d4e75fbf04cdb99b3c2ca4fb7f556 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f14d4e75fbf04cdb99b3c2ca4fb7f556" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39183 } }
I20260808 11:42:37.445967 22954 sys_catalog.cc:564] T 00000000000000000000000000000000 P f14d4e75fbf04cdb99b3c2ca4fb7f556 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:42:37.447750 22963 sys_catalog.cc:455] T 00000000000000000000000000000000 P f14d4e75fbf04cdb99b3c2ca4fb7f556 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f14d4e75fbf04cdb99b3c2ca4fb7f556. Latest consensus state: current_term: 1 leader_uuid: "f14d4e75fbf04cdb99b3c2ca4fb7f556" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f14d4e75fbf04cdb99b3c2ca4fb7f556" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39183 } } }
I20260808 11:42:37.447779 22962 sys_catalog.cc:455] T 00000000000000000000000000000000 P f14d4e75fbf04cdb99b3c2ca4fb7f556 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f14d4e75fbf04cdb99b3c2ca4fb7f556" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f14d4e75fbf04cdb99b3c2ca4fb7f556" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39183 } } }
I20260808 11:42:37.448194 22962 sys_catalog.cc:458] T 00000000000000000000000000000000 P f14d4e75fbf04cdb99b3c2ca4fb7f556 [sys.catalog]: This master's current role is: LEADER
I20260808 11:42:37.448194 22963 sys_catalog.cc:458] T 00000000000000000000000000000000 P f14d4e75fbf04cdb99b3c2ca4fb7f556 [sys.catalog]: This master's current role is: LEADER
I20260808 11:42:37.451203 22969 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:42:37.455947 22969 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:42:37.465685 22969 catalog_manager.cc:1347] Generated new cluster ID: 27778056081a4f9e9931b342bfb7ff0f
I20260808 11:42:37.465788 22969 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:42:37.492182 22969 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:42:37.493412 22969 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:42:37.499986 22969 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f14d4e75fbf04cdb99b3c2ca4fb7f556: Generated new TSK 0
I20260808 11:42:37.500867 22969 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.76.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab.
Entry for principal kudu/127.24.76.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab.
I20260808 11:42:37.594203 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.65:0
--local_ip_for_outbound_sockets=127.24.76.65
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:39183
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:33009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:42:37.942787 22991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:42:37.943049 22991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:42:37.943186 22991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:42:37.950413 22991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:42:37.950510 22991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:42:37.950680 22991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.65
I20260808 11:42:37.960009 22991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:33009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.65.keytab
--principal=kudu/127.24.76.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:39183
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.65
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:42:37.961203 22991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:42:37.962519 22991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:42:37.975279 22996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:37.975248 23000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:37.975350 22997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:42:37.975940 22991 server_base.cc:1034] running on GCE node
I20260808 11:42:37.976554 22991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:42:37.977440 22991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:42:37.978619 22991 hybrid_clock.cc:648] HybridClock initialized: now 1786189357978571 us; error 47 us; skew 500 ppm
I20260808 11:42:37.978904 22991 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:42:37 dist-test-slave-tc2s krb5kdc[22829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189357, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:42:37.982013 22991 init.cc:377] Logged in from keytab as kudu/127.24.76.65@KRBTEST.COM (short username kudu)
I20260808 11:42:37.983500 22991 webserver.cc:460] Webserver started at http://127.24.76.65:42713/ using document root <none> and password file <none>
I20260808 11:42:37.984154 22991 fs_manager.cc:359] Metadata directory not provided
I20260808 11:42:37.984272 22991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:42:37.984579 22991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:42:37.987443 22991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-0/data/instance:
uuid: "1fdc697cf359450cba133b76de04dde4"
format_stamp: "Formatted at 2026-08-08 11:42:37 on dist-test-slave-tc2s"
server_key: "e540f55f62320291e60cae84bec5a38d"
server_key_iv: "969bbb8ad6c14ef2ec2e3a381ff43b05"
server_key_version: "encryptionkey@0"
I20260808 11:42:37.988191 22991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance:
uuid: "1fdc697cf359450cba133b76de04dde4"
format_stamp: "Formatted at 2026-08-08 11:42:37 on dist-test-slave-tc2s"
server_key: "e540f55f62320291e60cae84bec5a38d"
server_key_iv: "969bbb8ad6c14ef2ec2e3a381ff43b05"
server_key_version: "encryptionkey@0"
I20260808 11:42:37.993160 22991 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260808 11:42:37.996447 23016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:37.997737 22991 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 11:42:37.997881 22991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-0/wal
uuid: "1fdc697cf359450cba133b76de04dde4"
format_stamp: "Formatted at 2026-08-08 11:42:37 on dist-test-slave-tc2s"
server_key: "e540f55f62320291e60cae84bec5a38d"
server_key_iv: "969bbb8ad6c14ef2ec2e3a381ff43b05"
server_key_version: "encryptionkey@0"
I20260808 11:42:37.998113 22991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:42:38.020090 22991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:42:38.024181 22991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:42:38.024454 22991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:42:38.025391 22991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:42:38.026687 22991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:42:38.026772 22991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:38.026891 22991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:42:38.026947 22991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:38.057507 22991 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.65:35199
I20260808 11:42:38.057598 23200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.65:35199 every 8 connection(s)
I20260808 11:42:38.058817 22991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
I20260808 11:42:38.068161 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 22991
I20260808 11:42:38.068346 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance
I20260808 11:42:38.068908 24881 external_mini_cluster.cc:1428] Setting key cf6adf75481828bbcc2684ae94ef89a7
Aug 08 11:42:38 dist-test-slave-tc2s krb5kdc[22829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189357, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.65@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:42:38.079046 23206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:38.061276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:35761 (local address 127.24.76.126:39183)
0808 11:42:38.061692 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:38.061697 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:38.062647 (+ 950us) server_negotiation.cc:407] Connection header received
0808 11:42:38.064600 (+ 1953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:38.064606 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:38.064692 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:38.064786 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:38.066311 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:38.067052 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:38.068971 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:38.069505 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:38.069644 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:38.073321 (+ 3677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:38.073328 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:38.073333 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:42:38.073371 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:38.076225 (+ 2854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:38.076898 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:38.076903 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:38.076906 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:38.077051 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:38.077410 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:38.077415 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:38.077417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:38.077738 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:38.077873 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:38.078507 (+ 634us) server_negotiation.cc:299] Negotiation successful
0808 11:42:38.078715 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":216,"threads_started":1}
I20260808 11:42:38.079732 23207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:38.061741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:39183 (local address 127.24.76.65:35761)
0808 11:42:38.062504 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:38.062547 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:38.064344 (+ 1797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:38.065043 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:38.065052 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:38.065543 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:42:38.066147 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:38.066168 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:38.067204 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:38.067208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:38.068824 (+ 1616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:38.068839 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:38.070010 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:38.070019 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:38.070326 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:38.070998 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:42:38.071027 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:42:38.073100 (+ 2073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:42:38.076350 (+ 3250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:42:38.076357 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:42:38.076368 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:42:38.076772 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:42:38.077138 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:42:38.077141 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:42:38.077143 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:42:38.077307 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:42:38.077838 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:42:38.077842 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:42:38.078154 (+ 312us) client_negotiation.cc:769] Sending connection context
0808 11:42:38.078363 (+ 209us) client_negotiation.cc:240] Negotiation successful
0808 11:42:38.078717 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":302,"threads_started":1}
I20260808 11:42:38.081060 23203 heartbeater.cc:347] Connected to a master server at 127.24.76.126:39183
I20260808 11:42:38.081378 23203 heartbeater.cc:464] Registering TS with master...
I20260808 11:42:38.082190 23203 heartbeater.cc:511] Master 127.24.76.126:39183 requested a full tablet report, sending...
I20260808 11:42:38.084391 22898 ts_manager.cc:194] Registered new tserver with Master: 1fdc697cf359450cba133b76de04dde4 (127.24.76.65:35199)
I20260808 11:42:38.085877 22898 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.65@KRBTEST.COM'} at 127.24.76.65:35761
WARNING: no policy specified for kudu/127.24.76.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab.
Entry for principal kudu/127.24.76.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab.
I20260808 11:42:38.134601 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.66:0
--local_ip_for_outbound_sockets=127.24.76.66
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:39183
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:33009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:42:38.460928 23213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:42:38.461189 23213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:42:38.461314 23213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:42:38.468297 23213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:42:38.468396 23213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:42:38.468546 23213 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.66
I20260808 11:42:38.477912 23213 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:33009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.66.keytab
--principal=kudu/127.24.76.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:39183
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.66
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:42:38.479033 23213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:42:38.480247 23213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:42:38.492551 23224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:38.492601 23221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:38.492626 23222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:42:38.493369 23213 server_base.cc:1034] running on GCE node
I20260808 11:42:38.493949 23213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:42:38.494804 23213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:42:38.495996 23213 hybrid_clock.cc:648] HybridClock initialized: now 1786189358495951 us; error 34 us; skew 500 ppm
I20260808 11:42:38.496280 23213 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:42:38 dist-test-slave-tc2s krb5kdc[22829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189358, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:42:38.501927 23213 init.cc:377] Logged in from keytab as kudu/127.24.76.66@KRBTEST.COM (short username kudu)
I20260808 11:42:38.503255 23213 webserver.cc:460] Webserver started at http://127.24.76.66:34313/ using document root <none> and password file <none>
I20260808 11:42:38.503892 23213 fs_manager.cc:359] Metadata directory not provided
I20260808 11:42:38.504007 23213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:42:38.504297 23213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:42:38.507148 23213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-1/data/instance:
uuid: "62d8bf73091144f4858f6f1b3c8878ae"
format_stamp: "Formatted at 2026-08-08 11:42:38 on dist-test-slave-tc2s"
server_key: "f40b64f2b2c0bb6173edacfb25a44c4a"
server_key_iv: "a8886b907501e6f721e7e39bce1bd2aa"
server_key_version: "encryptionkey@0"
I20260808 11:42:38.507902 23213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance:
uuid: "62d8bf73091144f4858f6f1b3c8878ae"
format_stamp: "Formatted at 2026-08-08 11:42:38 on dist-test-slave-tc2s"
server_key: "f40b64f2b2c0bb6173edacfb25a44c4a"
server_key_iv: "a8886b907501e6f721e7e39bce1bd2aa"
server_key_version: "encryptionkey@0"
I20260808 11:42:38.512648 23213 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260808 11:42:38.515842 23234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:38.517000 23213 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 11:42:38.517148 23213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-1/wal
uuid: "62d8bf73091144f4858f6f1b3c8878ae"
format_stamp: "Formatted at 2026-08-08 11:42:38 on dist-test-slave-tc2s"
server_key: "f40b64f2b2c0bb6173edacfb25a44c4a"
server_key_iv: "a8886b907501e6f721e7e39bce1bd2aa"
server_key_version: "encryptionkey@0"
I20260808 11:42:38.517325 23213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:42:38.549726 23213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:42:38.553539 23213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:42:38.553794 23213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:42:38.554725 23213 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:42:38.556015 23213 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:42:38.556092 23213 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:38.556213 23213 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:42:38.556267 23213 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:38.585466 23213 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.66:40521
I20260808 11:42:38.585513 23407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.66:40521 every 8 connection(s)
I20260808 11:42:38.586710 23213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
I20260808 11:42:38.589324 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 23213
I20260808 11:42:38.589529 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance
I20260808 11:42:38.590020 24881 external_mini_cluster.cc:1428] Setting key de214ed898ea914b59c786d10f8e6660
Aug 08 11:42:38 dist-test-slave-tc2s krb5kdc[22829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189358, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.66@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:42:38.606495 23413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:38.589126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:32879 (local address 127.24.76.126:39183)
0808 11:42:38.589593 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:38.589600 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:38.590605 (+ 1005us) server_negotiation.cc:407] Connection header received
0808 11:42:38.592757 (+ 2152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:38.592763 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:38.592843 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:38.592941 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:38.594365 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:38.595280 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:38.596757 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:38.597291 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:38.597409 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:38.600696 (+ 3287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:38.600703 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:38.600707 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:42:38.600743 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:38.603684 (+ 2941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:38.604332 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:38.604336 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:38.604339 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:38.604502 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:38.604862 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:38.604867 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:38.604869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:38.605185 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:38.605316 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:38.605951 (+ 635us) server_negotiation.cc:299] Negotiation successful
0808 11:42:38.606156 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":208,"threads_started":1}
I20260808 11:42:38.607399 23415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:38.589609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:39183 (local address 127.24.76.66:32879)
0808 11:42:38.590407 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:38.590481 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:38.592508 (+ 2027us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:38.593131 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:38.593139 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:38.593630 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:42:38.594210 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:38.594227 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:38.595407 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:38.595411 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:38.596621 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:38.596633 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:38.597474 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:38.597479 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:38.597655 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:38.598267 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:42:38.598293 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:42:38.600489 (+ 2196us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:42:38.603807 (+ 3318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:42:38.603814 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:42:38.603824 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:42:38.604211 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:42:38.604591 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:42:38.604594 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:42:38.604596 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:42:38.604763 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:42:38.605296 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:42:38.605301 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:42:38.605593 (+ 292us) client_negotiation.cc:769] Sending connection context
0808 11:42:38.605793 (+ 200us) client_negotiation.cc:240] Negotiation successful
0808 11:42:38.606173 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":313,"threads_started":1}
I20260808 11:42:38.608691 23409 heartbeater.cc:347] Connected to a master server at 127.24.76.126:39183
I20260808 11:42:38.608961 23409 heartbeater.cc:464] Registering TS with master...
I20260808 11:42:38.609619 23409 heartbeater.cc:511] Master 127.24.76.126:39183 requested a full tablet report, sending...
I20260808 11:42:38.611001 22898 ts_manager.cc:194] Registered new tserver with Master: 62d8bf73091144f4858f6f1b3c8878ae (127.24.76.66:40521)
I20260808 11:42:38.611792 22898 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.66@KRBTEST.COM'} at 127.24.76.66:32879
WARNING: no policy specified for kudu/127.24.76.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab.
Entry for principal kudu/127.24.76.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab.
I20260808 11:42:38.655027 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.67:0
--local_ip_for_outbound_sockets=127.24.76.67
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:39183
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:33009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:42:38.982064 23419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:42:38.982311 23419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:42:38.982411 23419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:42:38.989406 23419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:42:38.989535 23419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:42:38.989666 23419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.67
I20260808 11:42:38.999048 23419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:33009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.67.keytab
--principal=kudu/127.24.76.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:39183
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.67
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:42:39.000259 23419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:42:39.001587 23419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:42:39.014037 23429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:39.014065 23430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:39.014459 23434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:42:39.014853 23419 server_base.cc:1034] running on GCE node
I20260808 11:42:39.015442 23419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:42:39.016283 23419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:42:39.017486 23419 hybrid_clock.cc:648] HybridClock initialized: now 1786189359017426 us; error 52 us; skew 500 ppm
I20260808 11:42:39.017773 23419 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:42:39 dist-test-slave-tc2s krb5kdc[22829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189359, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:42:39.020841 23419 init.cc:377] Logged in from keytab as kudu/127.24.76.67@KRBTEST.COM (short username kudu)
I20260808 11:42:39.022280 23419 webserver.cc:460] Webserver started at http://127.24.76.67:33231/ using document root <none> and password file <none>
I20260808 11:42:39.022908 23419 fs_manager.cc:359] Metadata directory not provided
I20260808 11:42:39.023008 23419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:42:39.023293 23419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:42:39.026160 23419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-2/data/instance:
uuid: "b659e090a421444c98905b281be14849"
format_stamp: "Formatted at 2026-08-08 11:42:39 on dist-test-slave-tc2s"
server_key: "8a0d2df42ff3706330b2a5b5dc467b0e"
server_key_iv: "1bc022d2b5a6f73041968ddacd7e0b0b"
server_key_version: "encryptionkey@0"
I20260808 11:42:39.026871 23419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance:
uuid: "b659e090a421444c98905b281be14849"
format_stamp: "Formatted at 2026-08-08 11:42:39 on dist-test-slave-tc2s"
server_key: "8a0d2df42ff3706330b2a5b5dc467b0e"
server_key_iv: "1bc022d2b5a6f73041968ddacd7e0b0b"
server_key_version: "encryptionkey@0"
I20260808 11:42:39.031607 23419 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 11:42:39.034729 23444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:39.035925 23419 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 11:42:39.036047 23419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-2/wal
uuid: "b659e090a421444c98905b281be14849"
format_stamp: "Formatted at 2026-08-08 11:42:39 on dist-test-slave-tc2s"
server_key: "8a0d2df42ff3706330b2a5b5dc467b0e"
server_key_iv: "1bc022d2b5a6f73041968ddacd7e0b0b"
server_key_version: "encryptionkey@0"
I20260808 11:42:39.036199 23419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:42:39.069418 23419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:42:39.073061 23419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:42:39.073287 23419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:42:39.074174 23419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:42:39.075395 23419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:42:39.075472 23419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:39.075567 23419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:42:39.075609 23419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:39.088594 23203 heartbeater.cc:503] Master 127.24.76.126:39183 was elected leader, sending a full tablet report...
I20260808 11:42:39.110584 23419 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.67:41487
I20260808 11:42:39.110643 23621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.67:41487 every 8 connection(s)
I20260808 11:42:39.111769 23419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
I20260808 11:42:39.121416 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 23419
I20260808 11:42:39.121595 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance
I20260808 11:42:39.122134 24881 external_mini_cluster.cc:1428] Setting key a02707de05d95a491a988f9ff66c5124
Aug 08 11:42:39 dist-test-slave-tc2s krb5kdc[22829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189359, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.67@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:42:39.130623 23626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:39.114098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:54793 (local address 127.24.76.126:39183)
0808 11:42:39.114560 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:39.114565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:39.115364 (+ 799us) server_negotiation.cc:407] Connection header received
0808 11:42:39.117182 (+ 1818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:39.117188 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:39.117270 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:39.117365 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:39.118902 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:39.119699 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:39.121339 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:39.121972 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:39.122084 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:39.125451 (+ 3367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:39.125456 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:39.125460 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:42:39.125492 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:39.128047 (+ 2555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:39.128744 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:39.128748 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:39.128750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:39.128890 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:39.129219 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:39.129222 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:39.129224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:39.129467 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:39.129594 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:39.130186 (+ 592us) server_negotiation.cc:299] Negotiation successful
0808 11:42:39.130341 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":260,"threads_started":1}
I20260808 11:42:39.131388 23628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:39.114559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:39183 (local address 127.24.76.67:54793)
0808 11:42:39.115209 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:39.115254 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:39.116921 (+ 1667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:39.117612 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:39.117620 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:39.118172 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:42:39.118725 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:39.118744 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:39.119863 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:39.119866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:39.121199 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:39.121213 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:39.122206 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:39.122215 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:39.122435 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:39.123095 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:42:39.123117 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:42:39.125264 (+ 2147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:42:39.128184 (+ 2920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:42:39.128193 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:42:39.128206 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:42:39.128645 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:42:39.128970 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:42:39.128973 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:42:39.128975 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:42:39.129144 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:42:39.129562 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:42:39.129567 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:42:39.129839 (+ 272us) client_negotiation.cc:769] Sending connection context
0808 11:42:39.130104 (+ 265us) client_negotiation.cc:240] Negotiation successful
0808 11:42:39.130437 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":268,"threads_started":1}
I20260808 11:42:39.132565 23622 heartbeater.cc:347] Connected to a master server at 127.24.76.126:39183
I20260808 11:42:39.132812 23622 heartbeater.cc:464] Registering TS with master...
I20260808 11:42:39.133486 23622 heartbeater.cc:511] Master 127.24.76.126:39183 requested a full tablet report, sending...
I20260808 11:42:39.134865 22898 ts_manager.cc:194] Registered new tserver with Master: b659e090a421444c98905b281be14849 (127.24.76.67:41487)
I20260808 11:42:39.135808 22898 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.76.67@KRBTEST.COM'} at 127.24.76.67:54793
I20260808 11:42:39.136947 24881 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:42:39.152699 23626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:39.141458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40572 (local address 127.24.76.126:39183)
0808 11:42:39.141592 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:39.141596 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:39.142017 (+ 421us) server_negotiation.cc:407] Connection header received
0808 11:42:39.142164 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:39.142167 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:39.142224 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:39.142290 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:39.143560 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:39.144249 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:39.145435 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:39.145928 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:39.146029 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:39.148174 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:39.148179 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:39.148183 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:42:39.148212 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:39.150625 (+ 2413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:39.151207 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:39.151211 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:39.151213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:39.151323 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:39.151606 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:39.151609 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:39.151611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:39.151905 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:39.152080 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:39.152332 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 11:42:39.152489 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 11:42:39.156219 22898 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40572:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:42:39.162963 22898 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:42:39.185252 23640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:39.173824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:35199 (local address 127.0.0.1:47220)
0808 11:42:39.175506 (+ 1682us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:39.175558 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:39.175725 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:39.178002 (+ 2277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:39.178011 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:39.178037 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:42:39.178415 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:39.178428 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:39.180396 (+ 1968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:39.180402 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:39.182381 (+ 1979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:39.182396 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:39.184147 (+ 1751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:39.184155 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:39.184361 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:39.184421 (+ 60us) client_negotiation.cc:769] Sending connection context
0808 11:42:39.184543 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 11:42:39.184743 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1419,"mutex_wait_us":6,"thread_start_us":1242,"threads_started":1}
I20260808 11:42:39.185446 23646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:39.174637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44564 (local address 127.24.76.66:40521)
0808 11:42:39.176371 (+ 1734us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:39.176381 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:39.176835 (+ 454us) server_negotiation.cc:407] Connection header received
0808 11:42:39.176964 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:39.176972 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:39.177213 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:39.177401 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:42:39.178365 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:39.179925 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:39.182703 (+ 2778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:39.184038 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:39.184709 (+ 671us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:39.184885 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:39.184967 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 11:42:39.185123 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1562,"thread_start_us":1415,"threads_started":1}
I20260808 11:42:39.186048 23644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:39.174024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47220 (local address 127.24.76.65:35199)
0808 11:42:39.176878 (+ 2854us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:39.176886 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:39.176915 (+ 29us) server_negotiation.cc:407] Connection header received
0808 11:42:39.177524 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:39.177532 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:39.177747 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:39.178599 (+ 852us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:42:39.178835 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:39.180261 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:39.182534 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:39.183930 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:39.184074 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:39.184302 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:39.184604 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 11:42:39.185764 (+ 1160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2696,"thread_start_us":2539,"threads_started":1}
I20260808 11:42:39.186264 23648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:39.177230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57826 (local address 127.24.76.67:41487)
0808 11:42:39.178791 (+ 1561us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:39.178796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:39.178819 (+ 23us) server_negotiation.cc:407] Connection header received
0808 11:42:39.178885 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:39.178889 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:39.179035 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:39.179181 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:42:39.179694 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:39.181107 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:39.183304 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:39.184265 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:39.184379 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:39.184609 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:39.185345 (+ 736us) server_negotiation.cc:299] Negotiation successful
0808 11:42:39.185828 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1436,"thread_start_us":1273,"threads_started":1}
I20260808 11:42:39.186993 23645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:39.174065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:40521 (local address 127.0.0.1:44564)
0808 11:42:39.176574 (+ 2509us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:39.176596 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:39.176710 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:39.177667 (+ 957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:39.177676 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:39.177708 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:42:39.178144 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:39.178158 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:39.180062 (+ 1904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:39.180068 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:39.182538 (+ 2470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:39.182553 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:39.184279 (+ 1726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:39.184288 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:39.184444 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:39.184468 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 11:42:39.184526 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 11:42:39.185816 (+ 1290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2390,"mutex_wait_us":1162,"thread_start_us":2249,"threads_started":1}
I20260808 11:42:39.187039 23647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:39.176642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:41487 (local address 127.0.0.1:57826)
0808 11:42:39.177104 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:39.177126 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:39.177239 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:39.179196 (+ 1957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:39.179200 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:39.179224 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:42:39.179532 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:39.179542 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:39.181305 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:39.181311 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:39.183162 (+ 1851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:39.183174 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:39.184749 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:39.184755 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:39.184912 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:39.184942 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 11:42:39.185024 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 11:42:39.185141 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":1,"thread_start_us":864,"threads_started":1}
I20260808 11:42:39.189390 23098 tablet_service.cc:1467] Processing CreateTablet for tablet a82777e9995f477dab859e4f49cc6614 (DEFAULT_TABLE table=test-table [id=707df925e3244999a139c42e53ec9556]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:42:39.190400 23523 tablet_service.cc:1467] Processing CreateTablet for tablet a82777e9995f477dab859e4f49cc6614 (DEFAULT_TABLE table=test-table [id=707df925e3244999a139c42e53ec9556]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:42:39.190407 23304 tablet_service.cc:1467] Processing CreateTablet for tablet a82777e9995f477dab859e4f49cc6614 (DEFAULT_TABLE table=test-table [id=707df925e3244999a139c42e53ec9556]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:42:39.191162 23098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a82777e9995f477dab859e4f49cc6614. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:42:39.191732 23523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a82777e9995f477dab859e4f49cc6614. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:42:39.191735 23304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a82777e9995f477dab859e4f49cc6614. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:42:39.201941 23650 tablet_bootstrap.cc:492] T a82777e9995f477dab859e4f49cc6614 P 1fdc697cf359450cba133b76de04dde4: Bootstrap starting.
I20260808 11:42:39.205472 23652 tablet_bootstrap.cc:492] T a82777e9995f477dab859e4f49cc6614 P b659e090a421444c98905b281be14849: Bootstrap starting.
I20260808 11:42:39.206058 23650 tablet_bootstrap.cc:654] T a82777e9995f477dab859e4f49cc6614 P 1fdc697cf359450cba133b76de04dde4: Neither blocks nor log segments found. Creating new log.
I20260808 11:42:39.206247 23651 tablet_bootstrap.cc:492] T a82777e9995f477dab859e4f49cc6614 P 62d8bf73091144f4858f6f1b3c8878ae: Bootstrap starting.
I20260808 11:42:39.207382 23650 log.cc:824] T a82777e9995f477dab859e4f49cc6614 P 1fdc697cf359450cba133b76de04dde4: Log is configured to *not* fsync() on all Append() calls
I20260808 11:42:39.209009 23652 tablet_bootstrap.cc:654] T a82777e9995f477dab859e4f49cc6614 P b659e090a421444c98905b281be14849: Neither blocks nor log segments found. Creating new log.
I20260808 11:42:39.209923 23652 log.cc:824] T a82777e9995f477dab859e4f49cc6614 P b659e090a421444c98905b281be14849: Log is configured to *not* fsync() on all Append() calls
I20260808 11:42:39.210235 23651 tablet_bootstrap.cc:654] T a82777e9995f477dab859e4f49cc6614 P 62d8bf73091144f4858f6f1b3c8878ae: Neither blocks nor log segments found. Creating new log.
I20260808 11:42:39.210518 23650 tablet_bootstrap.cc:492] T a82777e9995f477dab859e4f49cc6614 P 1fdc697cf359450cba133b76de04dde4: No bootstrap required, opened a new log
I20260808 11:42:39.210783 23650 ts_tablet_manager.cc:1397] T a82777e9995f477dab859e4f49cc6614 P 1fdc697cf359450cba133b76de04dde4: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.005s
I20260808 11:42:39.211602 23651 log.cc:824] T a82777e9995f477dab859e4f49cc6614 P 62d8bf73091144f4858f6f1b3c8878ae: Log is configured to *not* fsync() on all Append() calls
I20260808 11:42:39.212729 23652 tablet_bootstrap.cc:492] T a82777e9995f477dab859e4f49cc6614 P b659e090a421444c98905b281be14849: No bootstrap required, opened a new log
I20260808 11:42:39.212980 23652 ts_tablet_manager.cc:1397] T a82777e9995f477dab859e4f49cc6614 P b659e090a421444c98905b281be14849: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260808 11:42:39.214457 23651 tablet_bootstrap.cc:492] T a82777e9995f477dab859e4f49cc6614 P 62d8bf73091144f4858f6f1b3c8878ae: No bootstrap required, opened a new log
I20260808 11:42:39.214699 23651 ts_tablet_manager.cc:1397] T a82777e9995f477dab859e4f49cc6614 P 62d8bf73091144f4858f6f1b3c8878ae: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.005s
I20260808 11:42:39.219534 23652 raft_consensus.cc:348] T a82777e9995f477dab859e4f49cc6614 P b659e090a421444c98905b281be14849 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdc697cf359450cba133b76de04dde4" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 35199 } } peers { permanent_uuid: "62d8bf73091144f4858f6f1b3c8878ae" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 40521 } } peers { permanent_uuid: "b659e090a421444c98905b281be14849" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41487 } }
I20260808 11:42:39.219862 23652 raft_consensus.cc:374] T a82777e9995f477dab859e4f49cc6614 P b659e090a421444c98905b281be14849 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:42:39.219962 23652 raft_consensus.cc:729] T a82777e9995f477dab859e4f49cc6614 P b659e090a421444c98905b281be14849 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b659e090a421444c98905b281be14849, State: Initialized, Role: FOLLOWER
I20260808 11:42:39.220501 23652 consensus_queue.cc:260] T a82777e9995f477dab859e4f49cc6614 P b659e090a421444c98905b281be14849 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdc697cf359450cba133b76de04dde4" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 35199 } } peers { permanent_uuid: "62d8bf73091144f4858f6f1b3c8878ae" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 40521 } } peers { permanent_uuid: "b659e090a421444c98905b281be14849" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41487 } }
I20260808 11:42:39.221529 23622 heartbeater.cc:503] Master 127.24.76.126:39183 was elected leader, sending a full tablet report...
I20260808 11:42:39.222232 23652 ts_tablet_manager.cc:1428] T a82777e9995f477dab859e4f49cc6614 P b659e090a421444c98905b281be14849: Time spent starting tablet: real 0.009s user 0.006s sys 0.005s
I20260808 11:42:39.221011 23650 raft_consensus.cc:348] T a82777e9995f477dab859e4f49cc6614 P 1fdc697cf359450cba133b76de04dde4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdc697cf359450cba133b76de04dde4" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 35199 } } peers { permanent_uuid: "62d8bf73091144f4858f6f1b3c8878ae" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 40521 } } peers { permanent_uuid: "b659e090a421444c98905b281be14849" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41487 } }
I20260808 11:42:39.222856 23650 raft_consensus.cc:374] T a82777e9995f477dab859e4f49cc6614 P 1fdc697cf359450cba133b76de04dde4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:42:39.222971 23650 raft_consensus.cc:729] T a82777e9995f477dab859e4f49cc6614 P 1fdc697cf359450cba133b76de04dde4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdc697cf359450cba133b76de04dde4, State: Initialized, Role: FOLLOWER
I20260808 11:42:39.223501 23650 consensus_queue.cc:260] T a82777e9995f477dab859e4f49cc6614 P 1fdc697cf359450cba133b76de04dde4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdc697cf359450cba133b76de04dde4" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 35199 } } peers { permanent_uuid: "62d8bf73091144f4858f6f1b3c8878ae" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 40521 } } peers { permanent_uuid: "b659e090a421444c98905b281be14849" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41487 } }
I20260808 11:42:39.224434 23651 raft_consensus.cc:348] T a82777e9995f477dab859e4f49cc6614 P 62d8bf73091144f4858f6f1b3c8878ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdc697cf359450cba133b76de04dde4" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 35199 } } peers { permanent_uuid: "62d8bf73091144f4858f6f1b3c8878ae" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 40521 } } peers { permanent_uuid: "b659e090a421444c98905b281be14849" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41487 } }
I20260808 11:42:39.224825 23651 raft_consensus.cc:374] T a82777e9995f477dab859e4f49cc6614 P 62d8bf73091144f4858f6f1b3c8878ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:42:39.224912 23651 raft_consensus.cc:729] T a82777e9995f477dab859e4f49cc6614 P 62d8bf73091144f4858f6f1b3c8878ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62d8bf73091144f4858f6f1b3c8878ae, State: Initialized, Role: FOLLOWER
I20260808 11:42:39.225093 23650 ts_tablet_manager.cc:1428] T a82777e9995f477dab859e4f49cc6614 P 1fdc697cf359450cba133b76de04dde4: Time spent starting tablet: real 0.014s user 0.009s sys 0.002s
I20260808 11:42:39.225528 23651 consensus_queue.cc:260] T a82777e9995f477dab859e4f49cc6614 P 62d8bf73091144f4858f6f1b3c8878ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdc697cf359450cba133b76de04dde4" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 35199 } } peers { permanent_uuid: "62d8bf73091144f4858f6f1b3c8878ae" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 40521 } } peers { permanent_uuid: "b659e090a421444c98905b281be14849" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41487 } }
I20260808 11:42:39.227409 23409 heartbeater.cc:503] Master 127.24.76.126:39183 was elected leader, sending a full tablet report...
I20260808 11:42:39.228992 23651 ts_tablet_manager.cc:1428] T a82777e9995f477dab859e4f49cc6614 P 62d8bf73091144f4858f6f1b3c8878ae: Time spent starting tablet: real 0.014s user 0.010s sys 0.003s
W20260808 11:42:39.311101 23205 tablet.cc:2369] T a82777e9995f477dab859e4f49cc6614 P 1fdc697cf359450cba133b76de04dde4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:42:39.323586 23661 raft_consensus.cc:482] T a82777e9995f477dab859e4f49cc6614 P 1fdc697cf359450cba133b76de04dde4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:42:39.323885 23661 raft_consensus.cc:504] T a82777e9995f477dab859e4f49cc6614 P 1fdc697cf359450cba133b76de04dde4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdc697cf359450cba133b76de04dde4" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 35199 } } peers { permanent_uuid: "62d8bf73091144f4858f6f1b3c8878ae" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 40521 } } peers { permanent_uuid: "b659e090a421444c98905b281be14849" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41487 } }
I20260808 11:42:39.325580 23661 leader_election.cc:288] T a82777e9995f477dab859e4f49cc6614 P 1fdc697cf359450cba133b76de04dde4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62d8bf73091144f4858f6f1b3c8878ae (127.24.76.66:40521), b659e090a421444c98905b281be14849 (127.24.76.67:41487)
I20260808 11:42:39.332330 23646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:39.325956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:58085 (local address 127.24.76.66:40521)
0808 11:42:39.326197 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:39.326203 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:39.326458 (+ 255us) server_negotiation.cc:407] Connection header received
0808 11:42:39.326646 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:39.326651 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:39.326715 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:39.326798 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:42:39.327551 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:39.328518 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:39.330477 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:39.331422 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:39.331556 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:39.331610 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:39.331963 (+ 353us) server_negotiation.cc:299] Negotiation successful
0808 11:42:39.332085 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 11:42:39.332340 23664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:39.325769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:40521 (local address 127.24.76.65:58085)
0808 11:42:39.326359 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:39.326375 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:39.326512 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:39.327033 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:39.327039 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:39.327064 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:42:39.327419 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:39.327432 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:39.328658 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:39.328664 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:39.330319 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:39.330332 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:39.331634 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:39.331641 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:39.331817 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:39.331850 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 11:42:39.331937 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 11:42:39.332083 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":315,"threads_started":1}
I20260808 11:42:39.332816 23665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:39.326462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:41487 (local address 127.24.76.65:49267)
0808 11:42:39.327237 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:39.327255 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:39.327371 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:39.327697 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:39.327701 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:39.327726 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:42:39.328093 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:39.328104 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:39.329258 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:39.329262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:39.330812 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:39.330827 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:39.332070 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:39.332076 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:39.332252 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:39.332281 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 11:42:39.332389 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 11:42:39.332509 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":465,"threads_started":1}
I20260808 11:42:39.333276 23341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a82777e9995f477dab859e4f49cc6614" candidate_uuid: "1fdc697cf359450cba133b76de04dde4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62d8bf73091144f4858f6f1b3c8878ae" is_pre_election: true
I20260808 11:42:39.332795 23648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:39.326714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:49267 (local address 127.24.76.67:41487)
0808 11:42:39.326935 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:39.326941 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:39.327340 (+ 399us) server_negotiation.cc:407] Connection header received
0808 11:42:39.327494 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:39.327499 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:39.327581 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:39.327648 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:42:39.328217 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:39.329122 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:39.330961 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:39.331834 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:39.331984 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:39.332052 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:39.332426 (+ 374us) server_negotiation.cc:299] Negotiation successful
0808 11:42:39.332565 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 11:42:39.333786 23341 raft_consensus.cc:2454] T a82777e9995f477dab859e4f49cc6614 P 62d8bf73091144f4858f6f1b3c8878ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fdc697cf359450cba133b76de04dde4 in term 0.
I20260808 11:42:39.334437 23022 leader_election.cc:302] T a82777e9995f477dab859e4f49cc6614 P 1fdc697cf359450cba133b76de04dde4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdc697cf359450cba133b76de04dde4, 62d8bf73091144f4858f6f1b3c8878ae; no voters:
I20260808 11:42:39.334446 23556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a82777e9995f477dab859e4f49cc6614" candidate_uuid: "1fdc697cf359450cba133b76de04dde4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b659e090a421444c98905b281be14849" is_pre_election: true
I20260808 11:42:39.334786 23661 raft_consensus.cc:2781] T a82777e9995f477dab859e4f49cc6614 P 1fdc697cf359450cba133b76de04dde4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:42:39.334899 23661 raft_consensus.cc:482] T a82777e9995f477dab859e4f49cc6614 P 1fdc697cf359450cba133b76de04dde4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:42:39.334967 23661 raft_consensus.cc:3037] T a82777e9995f477dab859e4f49cc6614 P 1fdc697cf359450cba133b76de04dde4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:42:39.334923 23556 raft_consensus.cc:2454] T a82777e9995f477dab859e4f49cc6614 P b659e090a421444c98905b281be14849 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fdc697cf359450cba133b76de04dde4 in term 0.
I20260808 11:42:39.337241 23661 raft_consensus.cc:504] T a82777e9995f477dab859e4f49cc6614 P 1fdc697cf359450cba133b76de04dde4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdc697cf359450cba133b76de04dde4" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 35199 } } peers { permanent_uuid: "62d8bf73091144f4858f6f1b3c8878ae" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 40521 } } peers { permanent_uuid: "b659e090a421444c98905b281be14849" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41487 } }
I20260808 11:42:39.337788 23661 leader_election.cc:288] T a82777e9995f477dab859e4f49cc6614 P 1fdc697cf359450cba133b76de04dde4 [CANDIDATE]: Term 1 election: Requested vote from peers 62d8bf73091144f4858f6f1b3c8878ae (127.24.76.66:40521), b659e090a421444c98905b281be14849 (127.24.76.67:41487)
I20260808 11:42:39.338276 23341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a82777e9995f477dab859e4f49cc6614" candidate_uuid: "1fdc697cf359450cba133b76de04dde4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62d8bf73091144f4858f6f1b3c8878ae"
I20260808 11:42:39.338367 23556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a82777e9995f477dab859e4f49cc6614" candidate_uuid: "1fdc697cf359450cba133b76de04dde4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b659e090a421444c98905b281be14849"
I20260808 11:42:39.338497 23341 raft_consensus.cc:3037] T a82777e9995f477dab859e4f49cc6614 P 62d8bf73091144f4858f6f1b3c8878ae [term 0 FOLLOWER]: Advancing to term 1
W20260808 11:42:39.338562 23411 tablet.cc:2369] T a82777e9995f477dab859e4f49cc6614 P 62d8bf73091144f4858f6f1b3c8878ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:42:39.338572 23556 raft_consensus.cc:3037] T a82777e9995f477dab859e4f49cc6614 P b659e090a421444c98905b281be14849 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:42:39.340983 23341 raft_consensus.cc:2454] T a82777e9995f477dab859e4f49cc6614 P 62d8bf73091144f4858f6f1b3c8878ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fdc697cf359450cba133b76de04dde4 in term 1.
I20260808 11:42:39.341008 23556 raft_consensus.cc:2454] T a82777e9995f477dab859e4f49cc6614 P b659e090a421444c98905b281be14849 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fdc697cf359450cba133b76de04dde4 in term 1.
I20260808 11:42:39.341332 23022 leader_election.cc:302] T a82777e9995f477dab859e4f49cc6614 P 1fdc697cf359450cba133b76de04dde4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdc697cf359450cba133b76de04dde4, 62d8bf73091144f4858f6f1b3c8878ae; no voters:
I20260808 11:42:39.341586 23661 raft_consensus.cc:2781] T a82777e9995f477dab859e4f49cc6614 P 1fdc697cf359450cba133b76de04dde4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:42:39.342038 23661 raft_consensus.cc:686] T a82777e9995f477dab859e4f49cc6614 P 1fdc697cf359450cba133b76de04dde4 [term 1 LEADER]: Becoming Leader. State: Replica: 1fdc697cf359450cba133b76de04dde4, State: Running, Role: LEADER
I20260808 11:42:39.342489 23661 consensus_queue.cc:237] T a82777e9995f477dab859e4f49cc6614 P 1fdc697cf359450cba133b76de04dde4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdc697cf359450cba133b76de04dde4" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 35199 } } peers { permanent_uuid: "62d8bf73091144f4858f6f1b3c8878ae" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 40521 } } peers { permanent_uuid: "b659e090a421444c98905b281be14849" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41487 } }
I20260808 11:42:39.346689 22898 catalog_manager.cc:5495] T a82777e9995f477dab859e4f49cc6614 P 1fdc697cf359450cba133b76de04dde4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fdc697cf359450cba133b76de04dde4 (127.24.76.65). New cstate: current_term: 1 leader_uuid: "1fdc697cf359450cba133b76de04dde4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdc697cf359450cba133b76de04dde4" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 35199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62d8bf73091144f4858f6f1b3c8878ae" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 40521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b659e090a421444c98905b281be14849" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41487 } health_report { overall_health: UNKNOWN } } }
W20260808 11:42:39.363238 23624 tablet.cc:2369] T a82777e9995f477dab859e4f49cc6614 P b659e090a421444c98905b281be14849: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:42:39.372128 23644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:39.366814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47228 (local address 127.24.76.65:35199)
0808 11:42:39.367043 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:39.367049 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:39.367182 (+ 133us) server_negotiation.cc:407] Connection header received
0808 11:42:39.367386 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:39.367390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:39.367523 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:39.367668 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:42:39.368016 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:39.368699 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:39.370086 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:39.370483 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:39.370589 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:39.371076 (+ 487us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:42:39.371544 (+ 468us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:42:39.371673 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:39.371803 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 11:42:39.371887 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 11:42:39.379534 23556 raft_consensus.cc:1261] T a82777e9995f477dab859e4f49cc6614 P b659e090a421444c98905b281be14849 [term 1 FOLLOWER]: Refusing update from remote peer 1fdc697cf359450cba133b76de04dde4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:42:39.379614 23341 raft_consensus.cc:1261] T a82777e9995f477dab859e4f49cc6614 P 62d8bf73091144f4858f6f1b3c8878ae [term 1 FOLLOWER]: Refusing update from remote peer 1fdc697cf359450cba133b76de04dde4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:42:39.380364 23667 consensus_queue.cc:1035] T a82777e9995f477dab859e4f49cc6614 P 1fdc697cf359450cba133b76de04dde4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b659e090a421444c98905b281be14849" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:42:39.380669 23661 consensus_queue.cc:1035] T a82777e9995f477dab859e4f49cc6614 P 1fdc697cf359450cba133b76de04dde4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62d8bf73091144f4858f6f1b3c8878ae" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 40521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:42:39.403913 22898 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:42:39.404186 22898 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:42:39.409512 22898 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f14d4e75fbf04cdb99b3c2ca4fb7f556: Sending DeleteTablet for 3 replicas of tablet a82777e9995f477dab859e4f49cc6614
I20260808 11:42:39.410995 23523 tablet_service.cc:1514] Processing DeleteTablet for tablet a82777e9995f477dab859e4f49cc6614 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:42:39 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:57826
I20260808 11:42:39.411042 23304 tablet_service.cc:1514] Processing DeleteTablet for tablet a82777e9995f477dab859e4f49cc6614 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:42:39 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:44564
I20260808 11:42:39.412557 23098 tablet_service.cc:1514] Processing DeleteTablet for tablet a82777e9995f477dab859e4f49cc6614 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:42:39 UTC) from {username='kudu', principal='kudu/127.24.76.126@KRBTEST.COM'} at 127.0.0.1:47220
I20260808 11:42:39.415360 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 22991
I20260808 11:42:39.438010 23698 tablet_replica.cc:330] T a82777e9995f477dab859e4f49cc6614 P 1fdc697cf359450cba133b76de04dde4: stopping tablet replica
I20260808 11:42:39.438498 23698 raft_consensus.cc:2229] T a82777e9995f477dab859e4f49cc6614 P 1fdc697cf359450cba133b76de04dde4 [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:42:39.439201 23698 raft_consensus.cc:2258] T a82777e9995f477dab859e4f49cc6614 P 1fdc697cf359450cba133b76de04dde4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:42:39.439888 23697 tablet_replica.cc:330] T a82777e9995f477dab859e4f49cc6614 P 62d8bf73091144f4858f6f1b3c8878ae: stopping tablet replica
I20260808 11:42:39.439983 23695 tablet_replica.cc:330] T a82777e9995f477dab859e4f49cc6614 P b659e090a421444c98905b281be14849: stopping tablet replica
I20260808 11:42:39.440256 23697 raft_consensus.cc:2229] T a82777e9995f477dab859e4f49cc6614 P 62d8bf73091144f4858f6f1b3c8878ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:42:39.440428 23695 raft_consensus.cc:2229] T a82777e9995f477dab859e4f49cc6614 P b659e090a421444c98905b281be14849 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:42:39.440737 23697 raft_consensus.cc:2258] T a82777e9995f477dab859e4f49cc6614 P 62d8bf73091144f4858f6f1b3c8878ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:42:39.440907 23695 raft_consensus.cc:2258] T a82777e9995f477dab859e4f49cc6614 P b659e090a421444c98905b281be14849 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:42:39.443058 23698 ts_tablet_manager.cc:1905] T a82777e9995f477dab859e4f49cc6614 P 1fdc697cf359450cba133b76de04dde4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:42:39.443423 23695 ts_tablet_manager.cc:1905] T a82777e9995f477dab859e4f49cc6614 P b659e090a421444c98905b281be14849: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:42:39.447067 23697 ts_tablet_manager.cc:1905] T a82777e9995f477dab859e4f49cc6614 P 62d8bf73091144f4858f6f1b3c8878ae: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 08 11:42:39 dist-test-slave-tc2s krb5kdc[22829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.65@KRBTEST.COM
I20260808 11:42:39.453754 23697 ts_tablet_manager.cc:1918] T a82777e9995f477dab859e4f49cc6614 P 62d8bf73091144f4858f6f1b3c8878ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:42:39.453944 23697 log.cc:1196] T a82777e9995f477dab859e4f49cc6614 P 62d8bf73091144f4858f6f1b3c8878ae: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-1/wal/wals/a82777e9995f477dab859e4f49cc6614
I20260808 11:42:39.454526 23697 ts_tablet_manager.cc:1939] T a82777e9995f477dab859e4f49cc6614 P 62d8bf73091144f4858f6f1b3c8878ae: Deleting consensus metadata
I20260808 11:42:39.456380 22880 catalog_manager.cc:4841] TS 62d8bf73091144f4858f6f1b3c8878ae (127.24.76.66:40521): tablet a82777e9995f477dab859e4f49cc6614 (table test-table [id=707df925e3244999a139c42e53ec9556]) successfully deleted
I20260808 11:42:39.457391 23695 ts_tablet_manager.cc:1918] T a82777e9995f477dab859e4f49cc6614 P b659e090a421444c98905b281be14849: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:42:39.457556 23695 log.cc:1196] T a82777e9995f477dab859e4f49cc6614 P b659e090a421444c98905b281be14849: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-2/wal/wals/a82777e9995f477dab859e4f49cc6614
I20260808 11:42:39.458087 23695 ts_tablet_manager.cc:1939] T a82777e9995f477dab859e4f49cc6614 P b659e090a421444c98905b281be14849: Deleting consensus metadata
I20260808 11:42:39.459853 22880 catalog_manager.cc:4841] TS b659e090a421444c98905b281be14849 (127.24.76.67:41487): tablet a82777e9995f477dab859e4f49cc6614 (table test-table [id=707df925e3244999a139c42e53ec9556]) successfully deleted
I20260808 11:42:39.461354 23644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:39.416689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47244 (local address 127.24.76.65:35199)
0808 11:42:39.416863 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:39.416869 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:39.441490 (+ 24621us) server_negotiation.cc:407] Connection header received
0808 11:42:39.441785 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:39.441793 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:39.441875 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:39.442015 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:39.443956 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:39.445139 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:39.447044 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:39.447718 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:39.447808 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:39.453224 (+ 5416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:39.453239 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:39.453255 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:42:39.453300 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:39.457511 (+ 4211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:39.458597 (+ 1086us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:39.458606 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:39.458610 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:39.458815 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:39.459274 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:39.459278 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:39.459282 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:39.459739 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:39.460111 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:39.460744 (+ 633us) server_negotiation.cc:299] Negotiation successful
0808 11:42:39.460987 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 11:42:39.461619 23698 ts_tablet_manager.cc:1918] T a82777e9995f477dab859e4f49cc6614 P 1fdc697cf359450cba133b76de04dde4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:42:39.461804 23698 log.cc:1196] T a82777e9995f477dab859e4f49cc6614 P 1fdc697cf359450cba133b76de04dde4: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786189042529715-24881-0/minicluster-data/ts-0/wal/wals/a82777e9995f477dab859e4f49cc6614
I20260808 11:42:39.462333 23698 ts_tablet_manager.cc:1939] T a82777e9995f477dab859e4f49cc6614 P 1fdc697cf359450cba133b76de04dde4: Deleting consensus metadata
I20260808 11:42:39.463162 23193 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:42:39.564253 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 22991
I20260808 11:42:39.564399 22878 catalog_manager.cc:4841] TS 1fdc697cf359450cba133b76de04dde4 (127.24.76.65:35199): tablet a82777e9995f477dab859e4f49cc6614 (table test-table [id=707df925e3244999a139c42e53ec9556]) successfully deleted
I20260808 11:42:39.590945 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 23213
Aug 08 11:42:39 dist-test-slave-tc2s krb5kdc[22829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.66@KRBTEST.COM
I20260808 11:42:39.606976 23646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:39.591676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44568 (local address 127.24.76.66:40521)
0808 11:42:39.591933 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:39.591940 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:39.591961 (+ 21us) server_negotiation.cc:407] Connection header received
0808 11:42:39.592085 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:39.592091 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:39.592176 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:39.592269 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:39.593477 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:39.594656 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:39.595898 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:39.596509 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:39.596636 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:39.600895 (+ 4259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:39.600911 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:39.600928 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:42:39.600968 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:39.604575 (+ 3607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:39.605231 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:39.605238 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:39.605240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:39.605379 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:39.605662 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:39.605664 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:39.605666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:39.606107 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:39.606373 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:39.606564 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 11:42:39.606722 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260808 11:42:39.607601 23402 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:42:39.696368 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 23213
I20260808 11:42:39.721776 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 23419
Aug 08 11:42:39 dist-test-slave-tc2s krb5kdc[22829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.67@KRBTEST.COM
I20260808 11:42:39.735931 23648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:39.722568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57842 (local address 127.24.76.67:41487)
0808 11:42:39.722763 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:39.722769 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:39.722800 (+ 31us) server_negotiation.cc:407] Connection header received
0808 11:42:39.722874 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:39.722878 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:39.722955 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:39.723042 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:39.724359 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:39.725079 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:39.726501 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:39.726876 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:39.726973 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:39.731180 (+ 4207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:39.731194 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:39.731212 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:42:39.731248 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:39.733612 (+ 2364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:39.734309 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:39.734316 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:39.734318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:39.734480 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:39.734752 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:39.734755 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:39.734757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:39.735104 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:39.735322 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:39.735524 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 11:42:39.735681 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 11:42:39.736533 23615 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:42:39.823459 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 23419
I20260808 11:42:39.849290 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 22850
I20260808 11:42:39.850329 22945 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:42:39.958084 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 22850
2026-08-08T11:42:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4524 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T11:42:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:42:40Z Disabled control of system clock
I20260808 11:42:40.051986 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:42541
--webserver_interface=127.24.76.126
--webserver_port=0
--builtin_ntp_servers=127.24.76.84:46437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:42541
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 11:42:40.380260 23716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:42:40.380569 23716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:42:40.380661 23716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:42:40.387614 23716 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:42:40.387707 23716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:42:40.387753 23716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:42:40.387806 23716 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:42:40.387850 23716 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:42:40.396842 23716 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:46437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:42541
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:42541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:42:40.398068 23716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:42:40.399791 23716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:42:40.410689 23723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:40.410777 23725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:40.410943 23731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:42:40.411782 23716 server_base.cc:1034] running on GCE node
I20260808 11:42:40.412597 23716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:42:40.413915 23716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:42:40.415132 23716 hybrid_clock.cc:648] HybridClock initialized: now 1786189360415082 us; error 44 us; skew 500 ppm
I20260808 11:42:40.415493 23716 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:42:40.417608 23716 webserver.cc:460] Webserver started at http://127.24.76.126:45809/ using document root <none> and password file <none>
I20260808 11:42:40.418278 23716 fs_manager.cc:359] Metadata directory not provided
I20260808 11:42:40.418388 23716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:42:40.418685 23716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:42:40.421679 23716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "73d97a8db3d4493eb286133034bf10aa"
format_stamp: "Formatted at 2026-08-08 11:42:40 on dist-test-slave-tc2s"
server_key: "15065aa1abf88660e99c34a782013963"
server_key_iv: "392163fd0a03476fbdb7e976a871943a"
server_key_version: "encryptionkey@0"
I20260808 11:42:40.422525 23716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "73d97a8db3d4493eb286133034bf10aa"
format_stamp: "Formatted at 2026-08-08 11:42:40 on dist-test-slave-tc2s"
server_key: "15065aa1abf88660e99c34a782013963"
server_key_iv: "392163fd0a03476fbdb7e976a871943a"
server_key_version: "encryptionkey@0"
I20260808 11:42:40.427644 23716 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 11:42:40.430989 23742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:40.432202 23716 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 11:42:40.432382 23716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "73d97a8db3d4493eb286133034bf10aa"
format_stamp: "Formatted at 2026-08-08 11:42:40 on dist-test-slave-tc2s"
server_key: "15065aa1abf88660e99c34a782013963"
server_key_iv: "392163fd0a03476fbdb7e976a871943a"
server_key_version: "encryptionkey@0"
I20260808 11:42:40.432577 23716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:42:40.446202 23716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:42:40.446923 23716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:42:40.447177 23716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:42:40.464191 23716 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:42541
I20260808 11:42:40.464250 23830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:42541 every 8 connection(s)
I20260808 11:42:40.465422 23716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:42:40.468930 23831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:42:40.474584 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 23716
I20260808 11:42:40.474764 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:42:40.475363 24881 external_mini_cluster.cc:1428] Setting key 3f2c708b81d2ac4ac3b61e8da82b1349
I20260808 11:42:40.479913 23831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73d97a8db3d4493eb286133034bf10aa: Bootstrap starting.
I20260808 11:42:40.483140 23831 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73d97a8db3d4493eb286133034bf10aa: Neither blocks nor log segments found. Creating new log.
I20260808 11:42:40.484069 23831 log.cc:824] T 00000000000000000000000000000000 P 73d97a8db3d4493eb286133034bf10aa: Log is configured to *not* fsync() on all Append() calls
I20260808 11:42:40.487128 23831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73d97a8db3d4493eb286133034bf10aa: No bootstrap required, opened a new log
I20260808 11:42:40.488570 23835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:40.477385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38646 (local address 127.24.76.126:42541)
0808 11:42:40.478358 (+ 973us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:40.478368 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:40.478408 (+ 40us) server_negotiation.cc:407] Connection header received
0808 11:42:40.480093 (+ 1685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:40.480114 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:40.480537 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:40.480855 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:40.481454 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:40.482807 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:40.484123 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:40.485060 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:40.485242 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:40.486504 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:40.486511 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:40.486523 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:42:40.486998 (+ 475us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:40.487070 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:42:40.487094 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:40.487181 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:40.487416 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 11:42:40.487714 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":324,"threads_started":1}
I20260808 11:42:40.494992 23831 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73d97a8db3d4493eb286133034bf10aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73d97a8db3d4493eb286133034bf10aa" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 42541 } }
I20260808 11:42:40.495302 23831 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73d97a8db3d4493eb286133034bf10aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:42:40.495395 23831 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73d97a8db3d4493eb286133034bf10aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73d97a8db3d4493eb286133034bf10aa, State: Initialized, Role: FOLLOWER
I20260808 11:42:40.495883 23831 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73d97a8db3d4493eb286133034bf10aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73d97a8db3d4493eb286133034bf10aa" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 42541 } }
I20260808 11:42:40.496095 23831 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73d97a8db3d4493eb286133034bf10aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:42:40.496218 23831 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73d97a8db3d4493eb286133034bf10aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:42:40.496371 23831 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73d97a8db3d4493eb286133034bf10aa [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:42:40.498478 23831 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73d97a8db3d4493eb286133034bf10aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73d97a8db3d4493eb286133034bf10aa" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 42541 } }
I20260808 11:42:40.498941 23831 leader_election.cc:302] T 00000000000000000000000000000000 P 73d97a8db3d4493eb286133034bf10aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73d97a8db3d4493eb286133034bf10aa; no voters:
I20260808 11:42:40.499457 23831 leader_election.cc:288] T 00000000000000000000000000000000 P 73d97a8db3d4493eb286133034bf10aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:42:40.499591 23839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73d97a8db3d4493eb286133034bf10aa [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:42:40.500334 23839 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73d97a8db3d4493eb286133034bf10aa [term 1 LEADER]: Becoming Leader. State: Replica: 73d97a8db3d4493eb286133034bf10aa, State: Running, Role: LEADER
I20260808 11:42:40.500831 23839 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73d97a8db3d4493eb286133034bf10aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73d97a8db3d4493eb286133034bf10aa" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 42541 } }
I20260808 11:42:40.501286 23831 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73d97a8db3d4493eb286133034bf10aa [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:42:40.503521 23840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73d97a8db3d4493eb286133034bf10aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73d97a8db3d4493eb286133034bf10aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73d97a8db3d4493eb286133034bf10aa" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 42541 } } }
I20260808 11:42:40.503909 23840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73d97a8db3d4493eb286133034bf10aa [sys.catalog]: This master's current role is: LEADER
I20260808 11:42:40.504181 23841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73d97a8db3d4493eb286133034bf10aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73d97a8db3d4493eb286133034bf10aa. Latest consensus state: current_term: 1 leader_uuid: "73d97a8db3d4493eb286133034bf10aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73d97a8db3d4493eb286133034bf10aa" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 42541 } } }
I20260808 11:42:40.504477 23841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73d97a8db3d4493eb286133034bf10aa [sys.catalog]: This master's current role is: LEADER
I20260808 11:42:40.506709 23849 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:42:40.510807 23849 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:42:40.518896 23849 catalog_manager.cc:1347] Generated new cluster ID: aee0742aebd74db097d17f8f46130687
I20260808 11:42:40.518986 23849 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:42:40.527594 23849 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:42:40.529240 23849 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:42:40.541445 23849 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73d97a8db3d4493eb286133034bf10aa: Generated new TSK 0
I20260808 11:42:40.542260 23849 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:42:40.548229 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.65:0
--local_ip_for_outbound_sockets=127.24.76.65
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:42541
--builtin_ntp_servers=127.24.76.84:46437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 11:42:40.885349 23870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:42:40.885618 23870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:42:40.885726 23870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:42:40.892735 23870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:42:40.892822 23870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:42:40.892976 23870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.65
I20260808 11:42:40.901744 23870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:46437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.76.65
--webserver_port=0
--tserver_master_addrs=127.24.76.126:42541
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.65
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:42:40.902977 23870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:42:40.904729 23870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:42:40.917155 23883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:40.917086 23892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:40.917124 23888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:42:40.918388 23870 server_base.cc:1034] running on GCE node
I20260808 11:42:40.918967 23870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:42:40.919847 23870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:42:40.921056 23870 hybrid_clock.cc:648] HybridClock initialized: now 1786189360921015 us; error 39 us; skew 500 ppm
I20260808 11:42:40.921347 23870 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:42:40.923434 23870 webserver.cc:460] Webserver started at http://127.24.76.65:41525/ using document root <none> and password file <none>
I20260808 11:42:40.924072 23870 fs_manager.cc:359] Metadata directory not provided
I20260808 11:42:40.924173 23870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:42:40.924468 23870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:42:40.927261 23870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-0/data/instance:
uuid: "0c15108c7aa34784ba854ab25c403657"
format_stamp: "Formatted at 2026-08-08 11:42:40 on dist-test-slave-tc2s"
server_key: "5e769a2e98921a68390c65de36ebd3fa"
server_key_iv: "18dc1dc39586151e9462d46881b678bf"
server_key_version: "encryptionkey@0"
I20260808 11:42:40.928004 23870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance:
uuid: "0c15108c7aa34784ba854ab25c403657"
format_stamp: "Formatted at 2026-08-08 11:42:40 on dist-test-slave-tc2s"
server_key: "5e769a2e98921a68390c65de36ebd3fa"
server_key_iv: "18dc1dc39586151e9462d46881b678bf"
server_key_version: "encryptionkey@0"
I20260808 11:42:40.932864 23870 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 11:42:40.936149 23901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:40.937409 23870 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 11:42:40.937562 23870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-0/wal
uuid: "0c15108c7aa34784ba854ab25c403657"
format_stamp: "Formatted at 2026-08-08 11:42:40 on dist-test-slave-tc2s"
server_key: "5e769a2e98921a68390c65de36ebd3fa"
server_key_iv: "18dc1dc39586151e9462d46881b678bf"
server_key_version: "encryptionkey@0"
I20260808 11:42:40.937743 23870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:42:40.954437 23870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:42:40.955156 23870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:42:40.955413 23870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:42:40.956290 23870 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:42:40.957635 23870 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:42:40.957726 23870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:40.957837 23870 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:42:40.957937 23870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:40.988307 23870 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.65:44325
I20260808 11:42:40.988387 24102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.65:44325 every 8 connection(s)
I20260808 11:42:40.989584 23870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-0/data/info.pb
I20260808 11:42:40.990051 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 23870
I20260808 11:42:40.990208 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-0/wal/instance
I20260808 11:42:40.990754 24881 external_mini_cluster.cc:1428] Setting key 745cb004b2b8304213264ff41cc1f9d0
I20260808 11:42:40.993626 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.66:0
--local_ip_for_outbound_sockets=127.24.76.66
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:42541
--builtin_ntp_servers=127.24.76.84:46437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 11:42:41.002406 24106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:40.992046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.65:45229 (local address 127.24.76.126:42541)
0808 11:42:40.992452 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:40.992457 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:40.993922 (+ 1465us) server_negotiation.cc:407] Connection header received
0808 11:42:40.995892 (+ 1970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:40.995896 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:40.995976 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:40.996085 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:40.997146 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:40.998082 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:40.999638 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:41.000168 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:41.000297 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:41.001579 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:41.001585 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:41.001589 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:42:41.001610 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:41.001659 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:42:41.001674 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:41.001753 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:41.001953 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 11:42:41.002044 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":211,"threads_started":1}
I20260808 11:42:41.003304 24107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:40.992750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:42541 (local address 127.24.76.65:45229)
0808 11:42:40.993752 (+ 1002us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:40.993805 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:40.995630 (+ 1825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:40.996326 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:40.996334 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:40.996421 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:42:40.996982 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:40.996999 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:40.998253 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:40.998258 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:40.999492 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:40.999506 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:41.000424 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:41.000429 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:41.000643 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:41.001295 (+ 652us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:42:41.001298 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:42:41.001335 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:42:41.001407 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:42:41.001766 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:42:41.001774 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:42:41.001780 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 11:42:41.001939 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 11:42:41.002247 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":298,"threads_started":1}
I20260808 11:42:41.004669 24104 heartbeater.cc:347] Connected to a master server at 127.24.76.126:42541
I20260808 11:42:41.004949 24104 heartbeater.cc:464] Registering TS with master...
I20260808 11:42:41.005654 24104 heartbeater.cc:511] Master 127.24.76.126:42541 requested a full tablet report, sending...
I20260808 11:42:41.007766 23769 ts_manager.cc:194] Registered new tserver with Master: 0c15108c7aa34784ba854ab25c403657 (127.24.76.65:44325)
I20260808 11:42:41.009207 23769 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.76.65:45229
W20260808 11:42:41.324457 24110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:42:41.324724 24110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:42:41.324852 24110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:42:41.331997 24110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:42:41.332095 24110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:42:41.332238 24110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.66
I20260808 11:42:41.341113 24110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:46437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.76.66
--webserver_port=0
--tserver_master_addrs=127.24.76.126:42541
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.66
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:42:41.342422 24110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:42:41.344126 24110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:42:41.357079 24115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:41.357136 24117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:41.357106 24119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:42:41.357722 24110 server_base.cc:1034] running on GCE node
I20260808 11:42:41.358381 24110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:42:41.359241 24110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:42:41.360431 24110 hybrid_clock.cc:648] HybridClock initialized: now 1786189361360385 us; error 42 us; skew 500 ppm
I20260808 11:42:41.360698 24110 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:42:41.362797 24110 webserver.cc:460] Webserver started at http://127.24.76.66:35985/ using document root <none> and password file <none>
I20260808 11:42:41.363392 24110 fs_manager.cc:359] Metadata directory not provided
I20260808 11:42:41.363476 24110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:42:41.363739 24110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:42:41.366498 24110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-1/data/instance:
uuid: "0ef066e7736743e0927b849a3b9ca46e"
format_stamp: "Formatted at 2026-08-08 11:42:41 on dist-test-slave-tc2s"
server_key: "faaa791a61b6e09a19a4cec721e6b143"
server_key_iv: "9c9882725fa252ee1f39d41909364905"
server_key_version: "encryptionkey@0"
I20260808 11:42:41.367204 24110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance:
uuid: "0ef066e7736743e0927b849a3b9ca46e"
format_stamp: "Formatted at 2026-08-08 11:42:41 on dist-test-slave-tc2s"
server_key: "faaa791a61b6e09a19a4cec721e6b143"
server_key_iv: "9c9882725fa252ee1f39d41909364905"
server_key_version: "encryptionkey@0"
I20260808 11:42:41.371891 24110 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260808 11:42:41.375027 24136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:41.376169 24110 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 11:42:41.376302 24110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-1/wal
uuid: "0ef066e7736743e0927b849a3b9ca46e"
format_stamp: "Formatted at 2026-08-08 11:42:41 on dist-test-slave-tc2s"
server_key: "faaa791a61b6e09a19a4cec721e6b143"
server_key_iv: "9c9882725fa252ee1f39d41909364905"
server_key_version: "encryptionkey@0"
I20260808 11:42:41.376475 24110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:42:41.390879 24110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:42:41.391556 24110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:42:41.391798 24110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:42:41.392632 24110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:42:41.393934 24110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:42:41.394027 24110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:41.394104 24110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:42:41.394156 24110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:41.425873 24110 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.66:33753
I20260808 11:42:41.425951 24328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.66:33753 every 8 connection(s)
I20260808 11:42:41.427158 24110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-1/data/info.pb
I20260808 11:42:41.433738 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 24110
I20260808 11:42:41.433941 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-1/wal/instance
I20260808 11:42:41.434479 24881 external_mini_cluster.cc:1428] Setting key d08053304b9ccab0338ee4ed0bcc9b69
I20260808 11:42:41.437088 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.76.67:0
--local_ip_for_outbound_sockets=127.24.76.67
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:42541
--builtin_ntp_servers=127.24.76.84:46437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 11:42:41.438925 24106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:41.429562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:34129 (local address 127.24.76.126:42541)
0808 11:42:41.429764 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:41.429770 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:41.431144 (+ 1374us) server_negotiation.cc:407] Connection header received
0808 11:42:41.432792 (+ 1648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:41.432796 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:41.432859 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:41.432950 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:41.433934 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:41.434779 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:41.436355 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:41.436787 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:41.436892 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:41.438167 (+ 1275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:41.438172 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:41.438175 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:42:41.438191 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:41.438222 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:42:41.438233 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:41.438303 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:41.438513 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 11:42:41.438609 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 11:42:41.439770 24333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:41.430137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:42541 (local address 127.24.76.66:34129)
0808 11:42:41.430995 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:41.431031 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:41.432574 (+ 1543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:41.433128 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:41.433135 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:41.433216 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:42:41.433736 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:41.433752 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:41.434937 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:41.434942 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:41.436220 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:41.436232 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:41.437001 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:41.437010 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:41.437244 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:41.437849 (+ 605us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:42:41.437852 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:42:41.437907 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:42:41.437996 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:42:41.438315 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:42:41.438327 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:42:41.438335 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 11:42:41.438479 (+ 144us) client_negotiation.cc:240] Negotiation successful
0808 11:42:41.438835 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":254,"threads_started":1}
I20260808 11:42:41.440997 24331 heartbeater.cc:347] Connected to a master server at 127.24.76.126:42541
I20260808 11:42:41.441285 24331 heartbeater.cc:464] Registering TS with master...
I20260808 11:42:41.442138 24331 heartbeater.cc:511] Master 127.24.76.126:42541 requested a full tablet report, sending...
I20260808 11:42:41.443507 23769 ts_manager.cc:194] Registered new tserver with Master: 0ef066e7736743e0927b849a3b9ca46e (127.24.76.66:33753)
I20260808 11:42:41.444257 23769 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.76.66:34129
W20260808 11:42:41.764909 24336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:42:41.765177 24336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:42:41.765303 24336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:42:41.772302 24336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:42:41.772403 24336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:42:41.772549 24336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.76.67
I20260808 11:42:41.781279 24336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:46437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.76.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.76.67
--webserver_port=0
--tserver_master_addrs=127.24.76.126:42541
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.76.67
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:42:41.782506 24336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:42:41.784197 24336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:42:41.796377 24343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:41.796416 24342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:41.796461 24347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:42:41.797088 24336 server_base.cc:1034] running on GCE node
I20260808 11:42:41.797672 24336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:42:41.798571 24336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:42:41.799769 24336 hybrid_clock.cc:648] HybridClock initialized: now 1786189361799721 us; error 44 us; skew 500 ppm
I20260808 11:42:41.800064 24336 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 11:42:41.802248 24336 webserver.cc:460] Webserver started at http://127.24.76.67:46745/ using document root <none> and password file <none>
I20260808 11:42:41.802893 24336 fs_manager.cc:359] Metadata directory not provided
I20260808 11:42:41.802994 24336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:42:41.803296 24336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:42:41.806089 24336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-2/data/instance:
uuid: "5e5b3b4d1dad4c549436501eecbf81ce"
format_stamp: "Formatted at 2026-08-08 11:42:41 on dist-test-slave-tc2s"
server_key: "71670c95b0b2a7d9a3058b3ac9604dad"
server_key_iv: "d379e7bfea314356149dd7c2bacb5fa4"
server_key_version: "encryptionkey@0"
I20260808 11:42:41.806898 24336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance:
uuid: "5e5b3b4d1dad4c549436501eecbf81ce"
format_stamp: "Formatted at 2026-08-08 11:42:41 on dist-test-slave-tc2s"
server_key: "71670c95b0b2a7d9a3058b3ac9604dad"
server_key_iv: "d379e7bfea314356149dd7c2bacb5fa4"
server_key_version: "encryptionkey@0"
I20260808 11:42:41.811864 24336 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 11:42:41.815136 24358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:41.816324 24336 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.004s sys 0.000s
I20260808 11:42:41.816473 24336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-2/wal
uuid: "5e5b3b4d1dad4c549436501eecbf81ce"
format_stamp: "Formatted at 2026-08-08 11:42:41 on dist-test-slave-tc2s"
server_key: "71670c95b0b2a7d9a3058b3ac9604dad"
server_key_iv: "d379e7bfea314356149dd7c2bacb5fa4"
server_key_version: "encryptionkey@0"
I20260808 11:42:41.816669 24336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:42:41.828953 24336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:42:41.829677 24336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:42:41.829985 24336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:42:41.830870 24336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 11:42:41.832239 24336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 11:42:41.832334 24336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:41.832444 24336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 11:42:41.832521 24336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:41.862702 24336 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.67:40657
I20260808 11:42:41.862752 24556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.67:40657 every 8 connection(s)
I20260808 11:42:41.863996 24336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-2/data/info.pb
I20260808 11:42:41.867085 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 24336
I20260808 11:42:41.867231 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-2/wal/instance
I20260808 11:42:41.867807 24881 external_mini_cluster.cc:1428] Setting key 5b4d26bf9a988df3892fa110e34a6787
I20260808 11:42:41.876519 24106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:41.866499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.67:42947 (local address 127.24.76.126:42541)
0808 11:42:41.866734 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:41.866740 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:41.868347 (+ 1607us) server_negotiation.cc:407] Connection header received
0808 11:42:41.870528 (+ 2181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:41.870534 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:41.870624 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:41.870714 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:41.871819 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:41.872555 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:41.874119 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:41.874552 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:41.874649 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:41.875832 (+ 1183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:41.875837 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:41.875840 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:42:41.875857 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:41.875890 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:42:41.875901 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:41.875968 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:41.876197 (+ 229us) server_negotiation.cc:299] Negotiation successful
0808 11:42:41.876276 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 11:42:41.877406 24561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:41.867026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:42541 (local address 127.24.76.67:42947)
0808 11:42:41.868163 (+ 1137us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:41.868221 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:41.870240 (+ 2019us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:41.870929 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:41.870937 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:41.871017 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:42:41.871620 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:41.871638 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:41.872699 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:41.872703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:41.873990 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:41.874003 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:41.874746 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:41.874751 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:41.874959 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:41.875570 (+ 611us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 11:42:41.875572 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:42:41.875608 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 11:42:41.875679 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:42:41.876006 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:42:41.876015 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:42:41.876022 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 11:42:41.876169 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 11:42:41.876437 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":385,"threads_started":1}
I20260808 11:42:41.878748 24558 heartbeater.cc:347] Connected to a master server at 127.24.76.126:42541
I20260808 11:42:41.879030 24558 heartbeater.cc:464] Registering TS with master...
I20260808 11:42:41.879766 24558 heartbeater.cc:511] Master 127.24.76.126:42541 requested a full tablet report, sending...
I20260808 11:42:41.881191 23769 ts_manager.cc:194] Registered new tserver with Master: 5e5b3b4d1dad4c549436501eecbf81ce (127.24.76.67:40657)
I20260808 11:42:41.882009 23769 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.76.67:42947
I20260808 11:42:41.882884 24881 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 11:42:41.892930 24106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:41.887269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38656 (local address 127.24.76.126:42541)
0808 11:42:41.887497 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:41.887503 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:41.887960 (+ 457us) server_negotiation.cc:407] Connection header received
0808 11:42:41.888122 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:41.888125 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:41.888195 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:41.888274 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:41.888786 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:41.889463 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:41.890856 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:41.891236 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:41.891344 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:41.892265 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:41.892270 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:41.892273 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:42:41.892289 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:41.892320 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:42:41.892332 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:41.892416 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:41.892553 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 11:42:41.892641 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 11:42:41.896384 23768 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38656:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 11:42:41.903532 23768 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 11:42:41.927587 24580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:41.915511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.66:33753 (local address 127.0.0.1:46454)
0808 11:42:41.918412 (+ 2901us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:41.918463 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:41.918644 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:41.919709 (+ 1065us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:41.919714 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:41.919742 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:42:41.920189 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:41.920201 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:41.922092 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:41.922097 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:41.924662 (+ 2565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:41.924677 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:41.926449 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:41.926455 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:41.926782 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:41.926852 (+ 70us) client_negotiation.cc:769] Sending connection context
0808 11:42:41.926955 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 11:42:41.927186 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2769,"mutex_wait_us":127,"spinlock_wait_cycles":6528,"thread_start_us":2627,"threads_started":1}
I20260808 11:42:41.927569 24581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:41.915651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46454 (local address 127.24.76.66:33753)
0808 11:42:41.916497 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:41.916506 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:41.918828 (+ 2322us) server_negotiation.cc:407] Connection header received
0808 11:42:41.918946 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:41.918955 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:41.919178 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:41.919363 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:42:41.920361 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:41.921958 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:41.924908 (+ 2950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:41.925835 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:41.925991 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:41.926316 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:41.927024 (+ 708us) server_negotiation.cc:299] Negotiation successful
0808 11:42:41.927185 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":525,"threads_started":1}
I20260808 11:42:41.927621 24584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:41.917674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50414 (local address 127.24.76.65:44325)
0808 11:42:41.918441 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:41.918449 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:41.920104 (+ 1655us) server_negotiation.cc:407] Connection header received
0808 11:42:41.920293 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:41.920301 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:41.920523 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:41.920781 (+ 258us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:42:41.921239 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:41.922632 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:41.924513 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:41.925809 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:41.925971 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:41.926289 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:41.927128 (+ 839us) server_negotiation.cc:299] Negotiation successful
0808 11:42:41.927278 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":333,"threads_started":1}
I20260808 11:42:41.928030 24583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:41.916117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58056 (local address 127.24.76.67:40657)
0808 11:42:41.916874 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:41.916882 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:41.918796 (+ 1914us) server_negotiation.cc:407] Connection header received
0808 11:42:41.918915 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:41.918923 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:41.919141 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:41.919308 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:42:41.920558 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:41.921934 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:41.924859 (+ 2925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:41.926252 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:41.926376 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:41.926629 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:41.927593 (+ 964us) server_negotiation.cc:299] Negotiation successful
0808 11:42:41.927745 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":431,"threads_started":1}
I20260808 11:42:41.928261 24578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:41.915497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:40657 (local address 127.0.0.1:58056)
0808 11:42:41.918412 (+ 2915us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:41.918464 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:41.918644 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:41.919398 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:41.919405 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:41.919435 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:42:41.920411 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:41.920424 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:41.922403 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:41.922412 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:41.924662 (+ 2250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:41.924677 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:41.926754 (+ 2077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:41.926760 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:41.926921 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:41.926950 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 11:42:41.927028 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 11:42:41.927176 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1835,"mutex_wait_us":43,"spinlock_wait_cycles":4864,"thread_start_us":1701,"threads_started":1}
I20260808 11:42:41.929476 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:41.917274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:44325 (local address 127.0.0.1:50414)
0808 11:42:41.919987 (+ 2713us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:41.920004 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:41.920148 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:41.920711 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:41.920715 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:41.920741 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:42:41.921077 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:41.921089 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:41.922767 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:41.922776 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:41.924329 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:41.924342 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:41.926440 (+ 2098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:41.926447 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:41.926799 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:41.926860 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 11:42:41.926957 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 11:42:41.929178 (+ 2221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2581,"mutex_wait_us":125,"thread_start_us":2409,"threads_started":1}
I20260808 11:42:41.931876 24449 tablet_service.cc:1467] Processing CreateTablet for tablet d91b981a116f49b88193e88e7de2f63f (DEFAULT_TABLE table=test-table [id=1931f36884474aa39230321b7e78e3ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:42:41.931878 24209 tablet_service.cc:1467] Processing CreateTablet for tablet d91b981a116f49b88193e88e7de2f63f (DEFAULT_TABLE table=test-table [id=1931f36884474aa39230321b7e78e3ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:42:41.933044 24449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d91b981a116f49b88193e88e7de2f63f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:42:41.932731 23988 tablet_service.cc:1467] Processing CreateTablet for tablet d91b981a116f49b88193e88e7de2f63f (DEFAULT_TABLE table=test-table [id=1931f36884474aa39230321b7e78e3ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 11:42:41.933244 24209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d91b981a116f49b88193e88e7de2f63f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:42:41.933956 23988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d91b981a116f49b88193e88e7de2f63f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:42:41.945174 24588 tablet_bootstrap.cc:492] T d91b981a116f49b88193e88e7de2f63f P 5e5b3b4d1dad4c549436501eecbf81ce: Bootstrap starting.
I20260808 11:42:41.947927 24589 tablet_bootstrap.cc:492] T d91b981a116f49b88193e88e7de2f63f P 0ef066e7736743e0927b849a3b9ca46e: Bootstrap starting.
I20260808 11:42:41.948592 24588 tablet_bootstrap.cc:654] T d91b981a116f49b88193e88e7de2f63f P 5e5b3b4d1dad4c549436501eecbf81ce: Neither blocks nor log segments found. Creating new log.
I20260808 11:42:41.949370 24590 tablet_bootstrap.cc:492] T d91b981a116f49b88193e88e7de2f63f P 0c15108c7aa34784ba854ab25c403657: Bootstrap starting.
I20260808 11:42:41.950006 24588 log.cc:824] T d91b981a116f49b88193e88e7de2f63f P 5e5b3b4d1dad4c549436501eecbf81ce: Log is configured to *not* fsync() on all Append() calls
I20260808 11:42:41.951025 24589 tablet_bootstrap.cc:654] T d91b981a116f49b88193e88e7de2f63f P 0ef066e7736743e0927b849a3b9ca46e: Neither blocks nor log segments found. Creating new log.
I20260808 11:42:41.952163 24589 log.cc:824] T d91b981a116f49b88193e88e7de2f63f P 0ef066e7736743e0927b849a3b9ca46e: Log is configured to *not* fsync() on all Append() calls
I20260808 11:42:41.952559 24590 tablet_bootstrap.cc:654] T d91b981a116f49b88193e88e7de2f63f P 0c15108c7aa34784ba854ab25c403657: Neither blocks nor log segments found. Creating new log.
I20260808 11:42:41.953649 24590 log.cc:824] T d91b981a116f49b88193e88e7de2f63f P 0c15108c7aa34784ba854ab25c403657: Log is configured to *not* fsync() on all Append() calls
I20260808 11:42:41.953960 24588 tablet_bootstrap.cc:492] T d91b981a116f49b88193e88e7de2f63f P 5e5b3b4d1dad4c549436501eecbf81ce: No bootstrap required, opened a new log
I20260808 11:42:41.954258 24588 ts_tablet_manager.cc:1397] T d91b981a116f49b88193e88e7de2f63f P 5e5b3b4d1dad4c549436501eecbf81ce: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260808 11:42:41.956106 24590 tablet_bootstrap.cc:492] T d91b981a116f49b88193e88e7de2f63f P 0c15108c7aa34784ba854ab25c403657: No bootstrap required, opened a new log
I20260808 11:42:41.956360 24590 ts_tablet_manager.cc:1397] T d91b981a116f49b88193e88e7de2f63f P 0c15108c7aa34784ba854ab25c403657: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 11:42:41.961189 24589 tablet_bootstrap.cc:492] T d91b981a116f49b88193e88e7de2f63f P 0ef066e7736743e0927b849a3b9ca46e: No bootstrap required, opened a new log
I20260808 11:42:41.961462 24589 ts_tablet_manager.cc:1397] T d91b981a116f49b88193e88e7de2f63f P 0ef066e7736743e0927b849a3b9ca46e: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.008s
I20260808 11:42:41.963665 24590 raft_consensus.cc:348] T d91b981a116f49b88193e88e7de2f63f P 0c15108c7aa34784ba854ab25c403657 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef066e7736743e0927b849a3b9ca46e" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33753 } } peers { permanent_uuid: "5e5b3b4d1dad4c549436501eecbf81ce" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 40657 } } peers { permanent_uuid: "0c15108c7aa34784ba854ab25c403657" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44325 } }
I20260808 11:42:41.964115 24590 raft_consensus.cc:374] T d91b981a116f49b88193e88e7de2f63f P 0c15108c7aa34784ba854ab25c403657 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:42:41.964222 24590 raft_consensus.cc:729] T d91b981a116f49b88193e88e7de2f63f P 0c15108c7aa34784ba854ab25c403657 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c15108c7aa34784ba854ab25c403657, State: Initialized, Role: FOLLOWER
I20260808 11:42:41.964377 24588 raft_consensus.cc:348] T d91b981a116f49b88193e88e7de2f63f P 5e5b3b4d1dad4c549436501eecbf81ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef066e7736743e0927b849a3b9ca46e" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33753 } } peers { permanent_uuid: "5e5b3b4d1dad4c549436501eecbf81ce" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 40657 } } peers { permanent_uuid: "0c15108c7aa34784ba854ab25c403657" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44325 } }
I20260808 11:42:41.964787 24588 raft_consensus.cc:374] T d91b981a116f49b88193e88e7de2f63f P 5e5b3b4d1dad4c549436501eecbf81ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:42:41.964888 24588 raft_consensus.cc:729] T d91b981a116f49b88193e88e7de2f63f P 5e5b3b4d1dad4c549436501eecbf81ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5b3b4d1dad4c549436501eecbf81ce, State: Initialized, Role: FOLLOWER
I20260808 11:42:41.964808 24590 consensus_queue.cc:260] T d91b981a116f49b88193e88e7de2f63f P 0c15108c7aa34784ba854ab25c403657 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef066e7736743e0927b849a3b9ca46e" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33753 } } peers { permanent_uuid: "5e5b3b4d1dad4c549436501eecbf81ce" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 40657 } } peers { permanent_uuid: "0c15108c7aa34784ba854ab25c403657" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44325 } }
I20260808 11:42:41.965524 24588 consensus_queue.cc:260] T d91b981a116f49b88193e88e7de2f63f P 5e5b3b4d1dad4c549436501eecbf81ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef066e7736743e0927b849a3b9ca46e" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33753 } } peers { permanent_uuid: "5e5b3b4d1dad4c549436501eecbf81ce" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 40657 } } peers { permanent_uuid: "0c15108c7aa34784ba854ab25c403657" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44325 } }
I20260808 11:42:41.966114 24104 heartbeater.cc:503] Master 127.24.76.126:42541 was elected leader, sending a full tablet report...
I20260808 11:42:41.966678 24590 ts_tablet_manager.cc:1428] T d91b981a116f49b88193e88e7de2f63f P 0c15108c7aa34784ba854ab25c403657: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260808 11:42:41.967134 24588 ts_tablet_manager.cc:1428] T d91b981a116f49b88193e88e7de2f63f P 5e5b3b4d1dad4c549436501eecbf81ce: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260808 11:42:41.967550 24558 heartbeater.cc:503] Master 127.24.76.126:42541 was elected leader, sending a full tablet report...
I20260808 11:42:41.973388 24589 raft_consensus.cc:348] T d91b981a116f49b88193e88e7de2f63f P 0ef066e7736743e0927b849a3b9ca46e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef066e7736743e0927b849a3b9ca46e" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33753 } } peers { permanent_uuid: "5e5b3b4d1dad4c549436501eecbf81ce" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 40657 } } peers { permanent_uuid: "0c15108c7aa34784ba854ab25c403657" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44325 } }
I20260808 11:42:41.973721 24589 raft_consensus.cc:374] T d91b981a116f49b88193e88e7de2f63f P 0ef066e7736743e0927b849a3b9ca46e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:42:41.973815 24589 raft_consensus.cc:729] T d91b981a116f49b88193e88e7de2f63f P 0ef066e7736743e0927b849a3b9ca46e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ef066e7736743e0927b849a3b9ca46e, State: Initialized, Role: FOLLOWER
I20260808 11:42:41.974344 24589 consensus_queue.cc:260] T d91b981a116f49b88193e88e7de2f63f P 0ef066e7736743e0927b849a3b9ca46e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef066e7736743e0927b849a3b9ca46e" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33753 } } peers { permanent_uuid: "5e5b3b4d1dad4c549436501eecbf81ce" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 40657 } } peers { permanent_uuid: "0c15108c7aa34784ba854ab25c403657" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44325 } }
I20260808 11:42:41.975262 24331 heartbeater.cc:503] Master 127.24.76.126:42541 was elected leader, sending a full tablet report...
I20260808 11:42:41.975625 24589 ts_tablet_manager.cc:1428] T d91b981a116f49b88193e88e7de2f63f P 0ef066e7736743e0927b849a3b9ca46e: Time spent starting tablet: real 0.014s user 0.008s sys 0.004s
W20260808 11:42:41.991851 24105 tablet.cc:2369] T d91b981a116f49b88193e88e7de2f63f P 0c15108c7aa34784ba854ab25c403657: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 11:42:42.115700 24559 tablet.cc:2369] T d91b981a116f49b88193e88e7de2f63f P 5e5b3b4d1dad4c549436501eecbf81ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:42:42.129004 24601 raft_consensus.cc:482] T d91b981a116f49b88193e88e7de2f63f P 0ef066e7736743e0927b849a3b9ca46e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 11:42:42.129267 24601 raft_consensus.cc:504] T d91b981a116f49b88193e88e7de2f63f P 0ef066e7736743e0927b849a3b9ca46e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef066e7736743e0927b849a3b9ca46e" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33753 } } peers { permanent_uuid: "5e5b3b4d1dad4c549436501eecbf81ce" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 40657 } } peers { permanent_uuid: "0c15108c7aa34784ba854ab25c403657" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44325 } }
I20260808 11:42:42.131021 24601 leader_election.cc:288] T d91b981a116f49b88193e88e7de2f63f P 0ef066e7736743e0927b849a3b9ca46e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5b3b4d1dad4c549436501eecbf81ce (127.24.76.67:40657), 0c15108c7aa34784ba854ab25c403657 (127.24.76.65:44325)
I20260808 11:42:42.137460 24584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:42.131501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:58075 (local address 127.24.76.65:44325)
0808 11:42:42.131688 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:42.131694 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:42.132304 (+ 610us) server_negotiation.cc:407] Connection header received
0808 11:42:42.132454 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:42.132458 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:42.132530 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:42.132621 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:42:42.133189 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:42.133874 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:42.135782 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:42.136556 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:42.136662 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:42.136711 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:42.137086 (+ 375us) server_negotiation.cc:299] Negotiation successful
0808 11:42:42.137217 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 11:42:42.137496 24604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:42.131294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:44325 (local address 127.24.76.66:58075)
0808 11:42:42.132187 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:42.132205 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:42.132332 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:42.132645 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:42.132650 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:42.132682 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:42:42.133041 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:42.133049 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:42.134036 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:42.134042 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:42.135588 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:42.135603 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:42.136727 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:42.136734 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:42.136894 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:42.136928 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 11:42:42.137017 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 11:42:42.137121 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":793,"mutex_wait_us":136,"thread_start_us":413,"threads_started":1}
I20260808 11:42:42.137667 24583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:42.131354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.76.66:43773 (local address 127.24.76.67:40657)
0808 11:42:42.131570 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:42.131576 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:42.132231 (+ 655us) server_negotiation.cc:407] Connection header received
0808 11:42:42.132410 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:42.132416 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:42.132489 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:42.132553 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 11:42:42.133167 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:42.133883 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:42.135790 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:42.136789 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:42.136939 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:42.137014 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:42.137331 (+ 317us) server_negotiation.cc:299] Negotiation successful
0808 11:42:42.137454 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 11:42:42.137878 24603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:42.131286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.67:40657 (local address 127.24.76.66:43773)
0808 11:42:42.132122 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:42.132138 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:42:42.132265 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:42:42.132589 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:42:42.132594 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:42:42.132618 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 11:42:42.133026 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:42.133035 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:42.134033 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:42.134037 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:42.135607 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:42:42.135619 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:42:42.136968 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:42:42.136973 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:42:42.137175 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:42.137206 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 11:42:42.137291 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 11:42:42.137403 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703,"mutex_wait_us":50,"thread_start_us":460,"threads_started":1}
I20260808 11:42:42.138438 24026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d91b981a116f49b88193e88e7de2f63f" candidate_uuid: "0ef066e7736743e0927b849a3b9ca46e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c15108c7aa34784ba854ab25c403657" is_pre_election: true
I20260808 11:42:42.138916 24026 raft_consensus.cc:2454] T d91b981a116f49b88193e88e7de2f63f P 0c15108c7aa34784ba854ab25c403657 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ef066e7736743e0927b849a3b9ca46e in term 0.
I20260808 11:42:42.138829 24487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d91b981a116f49b88193e88e7de2f63f" candidate_uuid: "0ef066e7736743e0927b849a3b9ca46e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5b3b4d1dad4c549436501eecbf81ce" is_pre_election: true
I20260808 11:42:42.139274 24487 raft_consensus.cc:2454] T d91b981a116f49b88193e88e7de2f63f P 5e5b3b4d1dad4c549436501eecbf81ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ef066e7736743e0927b849a3b9ca46e in term 0.
I20260808 11:42:42.139539 24140 leader_election.cc:302] T d91b981a116f49b88193e88e7de2f63f P 0ef066e7736743e0927b849a3b9ca46e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c15108c7aa34784ba854ab25c403657, 0ef066e7736743e0927b849a3b9ca46e; no voters:
I20260808 11:42:42.139921 24601 raft_consensus.cc:2781] T d91b981a116f49b88193e88e7de2f63f P 0ef066e7736743e0927b849a3b9ca46e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 11:42:42.140053 24601 raft_consensus.cc:482] T d91b981a116f49b88193e88e7de2f63f P 0ef066e7736743e0927b849a3b9ca46e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 11:42:42.140149 24601 raft_consensus.cc:3037] T d91b981a116f49b88193e88e7de2f63f P 0ef066e7736743e0927b849a3b9ca46e [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:42:42.142503 24601 raft_consensus.cc:504] T d91b981a116f49b88193e88e7de2f63f P 0ef066e7736743e0927b849a3b9ca46e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef066e7736743e0927b849a3b9ca46e" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33753 } } peers { permanent_uuid: "5e5b3b4d1dad4c549436501eecbf81ce" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 40657 } } peers { permanent_uuid: "0c15108c7aa34784ba854ab25c403657" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44325 } }
I20260808 11:42:42.143033 24601 leader_election.cc:288] T d91b981a116f49b88193e88e7de2f63f P 0ef066e7736743e0927b849a3b9ca46e [CANDIDATE]: Term 1 election: Requested vote from peers 5e5b3b4d1dad4c549436501eecbf81ce (127.24.76.67:40657), 0c15108c7aa34784ba854ab25c403657 (127.24.76.65:44325)
I20260808 11:42:42.143474 24026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d91b981a116f49b88193e88e7de2f63f" candidate_uuid: "0ef066e7736743e0927b849a3b9ca46e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c15108c7aa34784ba854ab25c403657"
I20260808 11:42:42.143477 24487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d91b981a116f49b88193e88e7de2f63f" candidate_uuid: "0ef066e7736743e0927b849a3b9ca46e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5b3b4d1dad4c549436501eecbf81ce"
I20260808 11:42:42.143630 24487 raft_consensus.cc:3037] T d91b981a116f49b88193e88e7de2f63f P 5e5b3b4d1dad4c549436501eecbf81ce [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:42:42.143646 24026 raft_consensus.cc:3037] T d91b981a116f49b88193e88e7de2f63f P 0c15108c7aa34784ba854ab25c403657 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:42:42.145707 24026 raft_consensus.cc:2454] T d91b981a116f49b88193e88e7de2f63f P 0c15108c7aa34784ba854ab25c403657 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ef066e7736743e0927b849a3b9ca46e in term 1.
I20260808 11:42:42.145735 24487 raft_consensus.cc:2454] T d91b981a116f49b88193e88e7de2f63f P 5e5b3b4d1dad4c549436501eecbf81ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ef066e7736743e0927b849a3b9ca46e in term 1.
I20260808 11:42:42.146093 24140 leader_election.cc:302] T d91b981a116f49b88193e88e7de2f63f P 0ef066e7736743e0927b849a3b9ca46e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c15108c7aa34784ba854ab25c403657, 0ef066e7736743e0927b849a3b9ca46e; no voters:
I20260808 11:42:42.146373 24601 raft_consensus.cc:2781] T d91b981a116f49b88193e88e7de2f63f P 0ef066e7736743e0927b849a3b9ca46e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:42:42.146772 24601 raft_consensus.cc:686] T d91b981a116f49b88193e88e7de2f63f P 0ef066e7736743e0927b849a3b9ca46e [term 1 LEADER]: Becoming Leader. State: Replica: 0ef066e7736743e0927b849a3b9ca46e, State: Running, Role: LEADER
I20260808 11:42:42.147244 24601 consensus_queue.cc:237] T d91b981a116f49b88193e88e7de2f63f P 0ef066e7736743e0927b849a3b9ca46e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef066e7736743e0927b849a3b9ca46e" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33753 } } peers { permanent_uuid: "5e5b3b4d1dad4c549436501eecbf81ce" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 40657 } } peers { permanent_uuid: "0c15108c7aa34784ba854ab25c403657" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44325 } }
I20260808 11:42:42.151496 23768 catalog_manager.cc:5495] T d91b981a116f49b88193e88e7de2f63f P 0ef066e7736743e0927b849a3b9ca46e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ef066e7736743e0927b849a3b9ca46e (127.24.76.66). New cstate: current_term: 1 leader_uuid: "0ef066e7736743e0927b849a3b9ca46e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ef066e7736743e0927b849a3b9ca46e" member_type: VOTER last_known_addr { host: "127.24.76.66" port: 33753 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5b3b4d1dad4c549436501eecbf81ce" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 40657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c15108c7aa34784ba854ab25c403657" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44325 } health_report { overall_health: UNKNOWN } } }
W20260808 11:42:42.179049 24332 tablet.cc:2369] T d91b981a116f49b88193e88e7de2f63f P 0ef066e7736743e0927b849a3b9ca46e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:42:42.216382 24581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:42.211217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46468 (local address 127.24.76.66:33753)
0808 11:42:42.211408 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:42.211414 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:42.211432 (+ 18us) server_negotiation.cc:407] Connection header received
0808 11:42:42.211663 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:42.211667 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:42.211752 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:42.211875 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:42:42.212247 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:42.212963 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:42.214369 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:42.214771 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:42.214880 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:42.215346 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:42:42.215840 (+ 494us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:42:42.215978 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:42.216076 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 11:42:42.216153 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 11:42:42.224112 24487 raft_consensus.cc:1261] T d91b981a116f49b88193e88e7de2f63f P 5e5b3b4d1dad4c549436501eecbf81ce [term 1 FOLLOWER]: Refusing update from remote peer 0ef066e7736743e0927b849a3b9ca46e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:42:42.224109 24026 raft_consensus.cc:1261] T d91b981a116f49b88193e88e7de2f63f P 0c15108c7aa34784ba854ab25c403657 [term 1 FOLLOWER]: Refusing update from remote peer 0ef066e7736743e0927b849a3b9ca46e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:42:42.225008 24601 consensus_queue.cc:1035] T d91b981a116f49b88193e88e7de2f63f P 0ef066e7736743e0927b849a3b9ca46e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5b3b4d1dad4c549436501eecbf81ce" member_type: VOTER last_known_addr { host: "127.24.76.67" port: 40657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:42:42.225282 24607 consensus_queue.cc:1035] T d91b981a116f49b88193e88e7de2f63f P 0ef066e7736743e0927b849a3b9ca46e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c15108c7aa34784ba854ab25c403657" member_type: VOTER last_known_addr { host: "127.24.76.65" port: 44325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:42:42.237838 24612 mvcc.cc:205] Tried to move back new op lower bound from 7316231627660750848 to 7316231627359117312. Current Snapshot: MvccSnapshot[applied={T|T < 7316231627660750848}]
I20260808 11:42:42.251113 23768 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38656:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:42:42.251339 23768 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38656:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 11:42:42.254786 23768 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 73d97a8db3d4493eb286133034bf10aa: Sending DeleteTablet for 3 replicas of tablet d91b981a116f49b88193e88e7de2f63f
I20260808 11:42:42.255781 24209 tablet_service.cc:1514] Processing DeleteTablet for tablet d91b981a116f49b88193e88e7de2f63f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:42:42 UTC) from {username='slave'} at 127.0.0.1:46454
I20260808 11:42:42.255770 24449 tablet_service.cc:1514] Processing DeleteTablet for tablet d91b981a116f49b88193e88e7de2f63f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:42:42 UTC) from {username='slave'} at 127.0.0.1:58056
I20260808 11:42:42.256471 23988 tablet_service.cc:1514] Processing DeleteTablet for tablet d91b981a116f49b88193e88e7de2f63f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 11:42:42 UTC) from {username='slave'} at 127.0.0.1:50414
I20260808 11:42:42.257001 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 23870
I20260808 11:42:42.266419 24628 tablet_replica.cc:330] T d91b981a116f49b88193e88e7de2f63f P 0c15108c7aa34784ba854ab25c403657: stopping tablet replica
I20260808 11:42:42.266937 24627 tablet_replica.cc:330] T d91b981a116f49b88193e88e7de2f63f P 0ef066e7736743e0927b849a3b9ca46e: stopping tablet replica
I20260808 11:42:42.267197 24625 tablet_replica.cc:330] T d91b981a116f49b88193e88e7de2f63f P 5e5b3b4d1dad4c549436501eecbf81ce: stopping tablet replica
I20260808 11:42:42.267339 24628 raft_consensus.cc:2229] T d91b981a116f49b88193e88e7de2f63f P 0c15108c7aa34784ba854ab25c403657 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:42:42.267593 24625 raft_consensus.cc:2229] T d91b981a116f49b88193e88e7de2f63f P 5e5b3b4d1dad4c549436501eecbf81ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 11:42:42.267783 24628 raft_consensus.cc:2258] T d91b981a116f49b88193e88e7de2f63f P 0c15108c7aa34784ba854ab25c403657 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:42:42.267976 24625 raft_consensus.cc:2258] T d91b981a116f49b88193e88e7de2f63f P 5e5b3b4d1dad4c549436501eecbf81ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:42:42.268342 24627 raft_consensus.cc:2229] T d91b981a116f49b88193e88e7de2f63f P 0ef066e7736743e0927b849a3b9ca46e [term 1 LEADER]: Raft consensus shutting down.
I20260808 11:42:42.268936 24627 raft_consensus.cc:2258] T d91b981a116f49b88193e88e7de2f63f P 0ef066e7736743e0927b849a3b9ca46e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 11:42:42.270721 24625 ts_tablet_manager.cc:1905] T d91b981a116f49b88193e88e7de2f63f P 5e5b3b4d1dad4c549436501eecbf81ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:42:42.270834 24627 ts_tablet_manager.cc:1905] T d91b981a116f49b88193e88e7de2f63f P 0ef066e7736743e0927b849a3b9ca46e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:42:42.271273 24628 ts_tablet_manager.cc:1905] T d91b981a116f49b88193e88e7de2f63f P 0c15108c7aa34784ba854ab25c403657: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 11:42:42.275156 24584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:42.258238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50424 (local address 127.24.76.65:44325)
0808 11:42:42.258419 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:42.258426 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:42.267313 (+ 8887us) server_negotiation.cc:407] Connection header received
0808 11:42:42.267755 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:42.267761 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:42.267870 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:42.267987 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:42.268471 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:42.269573 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:42.271462 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:42.272057 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:42.272148 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:42.273528 (+ 1380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:42.273538 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:42.273553 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:42:42.274259 (+ 706us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:42.274312 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:42:42.274335 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:42.274424 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:42.274674 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 11:42:42.274785 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 11:42:42.276414 24094 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:42:42.277504 24625 ts_tablet_manager.cc:1918] T d91b981a116f49b88193e88e7de2f63f P 5e5b3b4d1dad4c549436501eecbf81ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:42:42.277633 24627 ts_tablet_manager.cc:1918] T d91b981a116f49b88193e88e7de2f63f P 0ef066e7736743e0927b849a3b9ca46e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 11:42:42.277663 24625 log.cc:1196] T d91b981a116f49b88193e88e7de2f63f P 5e5b3b4d1dad4c549436501eecbf81ce: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-2/wal/wals/d91b981a116f49b88193e88e7de2f63f
I20260808 11:42:42.277778 24627 log.cc:1196] T d91b981a116f49b88193e88e7de2f63f P 0ef066e7736743e0927b849a3b9ca46e: Deleting WAL directory at /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786189042529715-24881-0/minicluster-data/ts-1/wal/wals/d91b981a116f49b88193e88e7de2f63f
I20260808 11:42:42.278242 24625 ts_tablet_manager.cc:1939] T d91b981a116f49b88193e88e7de2f63f P 5e5b3b4d1dad4c549436501eecbf81ce: Deleting consensus metadata
I20260808 11:42:42.278275 24627 ts_tablet_manager.cc:1939] T d91b981a116f49b88193e88e7de2f63f P 0ef066e7736743e0927b849a3b9ca46e: Deleting consensus metadata
I20260808 11:42:42.280144 23744 catalog_manager.cc:4841] TS 0ef066e7736743e0927b849a3b9ca46e (127.24.76.66:33753): tablet d91b981a116f49b88193e88e7de2f63f (table test-table [id=1931f36884474aa39230321b7e78e3ca]) successfully deleted
I20260808 11:42:42.280153 23748 catalog_manager.cc:4841] TS 5e5b3b4d1dad4c549436501eecbf81ce (127.24.76.67:40657): tablet d91b981a116f49b88193e88e7de2f63f (table test-table [id=1931f36884474aa39230321b7e78e3ca]) successfully deleted
I20260808 11:42:42.371043 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 23870
W20260808 11:42:42.404104 23750 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.76.65:44325 (error 108)
I20260808 11:42:42.404356 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 24110
I20260808 11:42:42.406226 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:42.405726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:44325 (local address 127.0.0.1:50430)
0808 11:42:42.405926 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:42.406049 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:44325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 11:42:42.406543 23750 catalog_manager.cc:4584] TS 0c15108c7aa34784ba854ab25c403657 (127.24.76.65:44325): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d91b981a116f49b88193e88e7de2f63f: Network error: Client connection negotiation failed: client connection to 127.24.76.65:44325: connect: Connection refused (error 111)
I20260808 11:42:42.412386 24581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:42.405427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46482 (local address 127.24.76.66:33753)
0808 11:42:42.405632 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:42.405638 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:42.405659 (+ 21us) server_negotiation.cc:407] Connection header received
0808 11:42:42.405731 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:42.405740 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:42.405862 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:42.405990 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:42.406504 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:42.407344 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:42.409206 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:42.409738 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:42.409831 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:42.411056 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:42.411065 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:42.411079 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:42:42.411657 (+ 578us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:42.411703 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:42:42.411732 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:42.411811 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:42.411978 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 11:42:42.412065 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 11:42:42.413077 24319 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:42:42.459982 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:42.459559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:44325 (local address 127.0.0.1:50444)
0808 11:42:42.459744 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:42.459837 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:44325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 11:42:42.460970 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:42.460619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:44325 (local address 127.0.0.1:50450)
0808 11:42:42.460765 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:42.460836 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:44325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 11:42:42.521591 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:42.521101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:44325 (local address 127.0.0.1:50452)
0808 11:42:42.521292 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:42.521393 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:44325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 11:42:42.522814 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:42.522446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:44325 (local address 127.0.0.1:50464)
0808 11:42:42.522595 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:42.522675 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:44325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 11:42:42.527254 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 24110
I20260808 11:42:42.555720 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 24336
I20260808 11:42:42.562036 24583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:42.556478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58068 (local address 127.24.76.67:40657)
0808 11:42:42.556658 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:42.556662 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:42.556681 (+ 19us) server_negotiation.cc:407] Connection header received
0808 11:42:42.556839 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:42.556843 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:42.556942 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:42.557031 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:42.557364 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:42.558143 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:42.559412 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:42.559788 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:42.559878 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:42.560782 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:42.560790 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:42.560805 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:42:42.561346 (+ 541us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:42.561391 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:42:42.561431 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:42.561522 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:42.561606 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 11:42:42.561691 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 11:42:42.562654 24545 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:42:42.603467 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:42.602996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:44325 (local address 127.0.0.1:50466)
0808 11:42:42.603187 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:42.603278 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:44325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 11:42:42.604584 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:42.604183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:44325 (local address 127.0.0.1:50470)
0808 11:42:42.604388 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:42.604470 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:44325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260808 11:42:42.649051 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 24336
I20260808 11:42:42.671178 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 23716
I20260808 11:42:42.672204 23825 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:42:42.778679 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:42.778013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.65:44325 (local address 127.0.0.1:36104)
0808 11:42:42.778343 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0808 11:42:42.778453 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.65:44325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186}
I20260808 11:42:42.778785 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 23716
2026-08-08T11:42:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2822 ms)
[----------] 6 tests from EncryptionPolicyTest (22082 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:42:42 dist-test-slave-tc2s krb5kdc[24643](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:42:42 dist-test-slave-tc2s krb5kdc[24643](info): set up 2 sockets
Aug 08 11:42:42 dist-test-slave-tc2s krb5kdc[24643](info): commencing operation
krb5kdc: starting...
W20260808 11:42:44.029088 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:42:44 dist-test-slave-tc2s krb5kdc[24643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:42:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:42:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 11:42:44.322067 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39717
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:42:44.653496 24667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:42:44.653759 24667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:42:44.653847 24667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:42:44.661023 24667 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:42:44.661124 24667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:42:44.661175 24667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:42:44.661233 24667 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:42:44.661280 24667 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:42:44.670954 24667 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39717
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:42:44.672106 24667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:42:44.673368 24667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:42:44.684695 24674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:44.684753 24679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:44.684799 24683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:42:44.685613 24667 server_base.cc:1034] running on GCE node
I20260808 11:42:44.686398 24667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:42:44.687716 24667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:42:44.688912 24667 hybrid_clock.cc:648] HybridClock initialized: now 1786189364688865 us; error 36 us; skew 500 ppm
I20260808 11:42:44.689240 24667 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:42:44 dist-test-slave-tc2s krb5kdc[24643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189364, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:42:44.696694 24667 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 11:42:44.698168 24667 webserver.cc:460] Webserver started at http://127.0.0.1:45241/ using document root <none> and password file <none>
I20260808 11:42:44.698851 24667 fs_manager.cc:359] Metadata directory not provided
I20260808 11:42:44.698962 24667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:42:44.699255 24667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:42:44.702260 24667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "594d2b4863084fe2bbd493fe851b3463"
format_stamp: "Formatted at 2026-08-08 11:42:44 on dist-test-slave-tc2s"
server_key: "2e5b23ddb73e12f877c1d94be62507bc"
server_key_iv: "73f41df024a7895a90726bca762f0100"
server_key_version: "encryptionkey@0"
I20260808 11:42:44.703038 24667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "594d2b4863084fe2bbd493fe851b3463"
format_stamp: "Formatted at 2026-08-08 11:42:44 on dist-test-slave-tc2s"
server_key: "2e5b23ddb73e12f877c1d94be62507bc"
server_key_iv: "73f41df024a7895a90726bca762f0100"
server_key_version: "encryptionkey@0"
I20260808 11:42:44.707933 24667 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 11:42:44.711264 24696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:44.712551 24667 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260808 11:42:44.712704 24667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "594d2b4863084fe2bbd493fe851b3463"
format_stamp: "Formatted at 2026-08-08 11:42:44 on dist-test-slave-tc2s"
server_key: "2e5b23ddb73e12f877c1d94be62507bc"
server_key_iv: "73f41df024a7895a90726bca762f0100"
server_key_version: "encryptionkey@0"
I20260808 11:42:44.712888 24667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:42:44.728999 24667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:42:44.733145 24667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:42:44.733399 24667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:42:44.749631 24667 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39717
I20260808 11:42:44.749672 24782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39717 every 8 connection(s)
I20260808 11:42:44.750950 24667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:42:44.754529 24783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:42:44.759892 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 24667
I20260808 11:42:44.760083 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:42:44.760702 24881 external_mini_cluster.cc:1428] Setting key 047109f79d1438d25debf361cc0f2d96
I20260808 11:42:44.765713 24783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 594d2b4863084fe2bbd493fe851b3463: Bootstrap starting.
I20260808 11:42:44.769341 24783 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 594d2b4863084fe2bbd493fe851b3463: Neither blocks nor log segments found. Creating new log.
I20260808 11:42:44.770516 24783 log.cc:824] T 00000000000000000000000000000000 P 594d2b4863084fe2bbd493fe851b3463: Log is configured to *not* fsync() on all Append() calls
I20260808 11:42:44.773458 24783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 594d2b4863084fe2bbd493fe851b3463: No bootstrap required, opened a new log
Aug 08 11:42:44 dist-test-slave-tc2s krb5kdc[24643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 11:42:44.782709 24783 raft_consensus.cc:348] T 00000000000000000000000000000000 P 594d2b4863084fe2bbd493fe851b3463 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "594d2b4863084fe2bbd493fe851b3463" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39717 } }
I20260808 11:42:44.783174 24783 raft_consensus.cc:374] T 00000000000000000000000000000000 P 594d2b4863084fe2bbd493fe851b3463 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:42:44.783288 24783 raft_consensus.cc:729] T 00000000000000000000000000000000 P 594d2b4863084fe2bbd493fe851b3463 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 594d2b4863084fe2bbd493fe851b3463, State: Initialized, Role: FOLLOWER
I20260808 11:42:44.783913 24783 consensus_queue.cc:260] T 00000000000000000000000000000000 P 594d2b4863084fe2bbd493fe851b3463 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "594d2b4863084fe2bbd493fe851b3463" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39717 } }
I20260808 11:42:44.784212 24783 raft_consensus.cc:388] T 00000000000000000000000000000000 P 594d2b4863084fe2bbd493fe851b3463 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:42:44.784353 24783 raft_consensus.cc:482] T 00000000000000000000000000000000 P 594d2b4863084fe2bbd493fe851b3463 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:42:44.784480 24789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:44.762603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51379 (local address 127.0.0.1:39717)
0808 11:42:44.763412 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:44.763426 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:44.763479 (+ 53us) server_negotiation.cc:407] Connection header received
0808 11:42:44.765742 (+ 2263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:44.765771 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:44.766265 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:44.766682 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:44.767959 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:44.769946 (+ 1987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:44.771342 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:44.772183 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:44.772353 (+ 170us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:44.776247 (+ 3894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:44.776259 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:44.776282 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:42:44.776336 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:44.780183 (+ 3847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:44.780919 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:44.780927 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:44.780935 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:44.781149 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:44.781537 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:44.781542 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:44.781545 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:44.782216 (+ 671us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:44.782533 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:44.782879 (+ 346us) server_negotiation.cc:299] Negotiation successful
0808 11:42:44.783208 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":302,"threads_started":1}
I20260808 11:42:44.784539 24783 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 594d2b4863084fe2bbd493fe851b3463 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:42:44.787321 24783 raft_consensus.cc:504] T 00000000000000000000000000000000 P 594d2b4863084fe2bbd493fe851b3463 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "594d2b4863084fe2bbd493fe851b3463" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39717 } }
I20260808 11:42:44.787838 24783 leader_election.cc:302] T 00000000000000000000000000000000 P 594d2b4863084fe2bbd493fe851b3463 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 594d2b4863084fe2bbd493fe851b3463; no voters:
I20260808 11:42:44.788460 24881 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:42:44.788473 24783 leader_election.cc:288] T 00000000000000000000000000000000 P 594d2b4863084fe2bbd493fe851b3463 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:42:44.788604 24792 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 594d2b4863084fe2bbd493fe851b3463 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:42:44.789170 24792 raft_consensus.cc:686] T 00000000000000000000000000000000 P 594d2b4863084fe2bbd493fe851b3463 [term 1 LEADER]: Becoming Leader. State: Replica: 594d2b4863084fe2bbd493fe851b3463, State: Running, Role: LEADER
I20260808 11:42:44.789760 24792 consensus_queue.cc:237] T 00000000000000000000000000000000 P 594d2b4863084fe2bbd493fe851b3463 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "594d2b4863084fe2bbd493fe851b3463" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39717 } }
I20260808 11:42:44.791563 24783 sys_catalog.cc:564] T 00000000000000000000000000000000 P 594d2b4863084fe2bbd493fe851b3463 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:42:44.792706 24795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 594d2b4863084fe2bbd493fe851b3463 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "594d2b4863084fe2bbd493fe851b3463" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "594d2b4863084fe2bbd493fe851b3463" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39717 } } }
I20260808 11:42:44.792695 24794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 594d2b4863084fe2bbd493fe851b3463 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 594d2b4863084fe2bbd493fe851b3463. Latest consensus state: current_term: 1 leader_uuid: "594d2b4863084fe2bbd493fe851b3463" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "594d2b4863084fe2bbd493fe851b3463" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39717 } } }
I20260808 11:42:44.793174 24794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 594d2b4863084fe2bbd493fe851b3463 [sys.catalog]: This master's current role is: LEADER
I20260808 11:42:44.793277 24795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 594d2b4863084fe2bbd493fe851b3463 [sys.catalog]: This master's current role is: LEADER
I20260808 11:42:44.806597 24803 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:42:44.812273 24803 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:42:44.825673 24803 catalog_manager.cc:1347] Generated new cluster ID: 8c4b495f98074bd1989c6f7db00f8469
I20260808 11:42:44.826468 24803 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:42:44.838183 24789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:44.808320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54695 (local address 127.0.0.1:39717)
0808 11:42:44.808499 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:44.808509 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:44.819206 (+ 10697us) server_negotiation.cc:407] Connection header received
0808 11:42:44.819712 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:44.819718 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:44.819806 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:44.819879 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:44.822031 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:44.823287 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:44.824506 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:44.826219 (+ 1713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:44.826348 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:44.829800 (+ 3452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:44.829808 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:44.829814 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:42:44.829857 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:44.834072 (+ 4215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:44.835170 (+ 1098us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:44.835176 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:44.835180 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:44.835370 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:44.836017 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:44.836023 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:44.836027 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:44.836475 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:44.836749 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:44.837520 (+ 771us) server_negotiation.cc:299] Negotiation successful
0808 11:42:44.837740 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 11:42:44.875250 24803 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:42:44.877120 24803 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:42:44.891803 24803 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 594d2b4863084fe2bbd493fe851b3463: Generated new TSK 0
I20260808 11:42:44.892783 24803 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:42:44.899402 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 24667
I20260808 11:42:44.900949 24777 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:42:44.992444 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 24667
W20260808 11:42:45.016624 24640 connection.cc:535] client connection to 127.0.0.1:39717 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39717): Connection reset by peer (error 104)
2026-08-08T11:42:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2205 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:42:45 dist-test-slave-tc2s krb5kdc[24833](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:42:45 dist-test-slave-tc2s krb5kdc[24833](info): set up 2 sockets
Aug 08 11:42:45 dist-test-slave-tc2s krb5kdc[24833](info): commencing operation
krb5kdc: starting...
W20260808 11:42:46.219431 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:42:46 dist-test-slave-tc2s krb5kdc[24833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:42:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:42:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 11:42:46.513434 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41705
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41705
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:42:46.851162 24859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:42:46.851449 24859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:42:46.851574 24859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:42:46.858655 24859 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:42:46.858760 24859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:42:46.858834 24859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:42:46.858922 24859 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:42:46.858987 24859 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:42:46.868594 24859 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41705
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:42:46.869868 24859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:42:46.871188 24859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:42:46.882413 24865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:46.882524 24871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:46.882558 24866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:42:46.883566 24859 server_base.cc:1034] running on GCE node
I20260808 11:42:46.884299 24859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:42:46.885671 24859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:42:46.886878 24859 hybrid_clock.cc:648] HybridClock initialized: now 1786189366886824 us; error 41 us; skew 500 ppm
I20260808 11:42:46.887212 24859 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:42:46 dist-test-slave-tc2s krb5kdc[24833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189366, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:42:46.890365 24859 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 11:42:46.891812 24859 webserver.cc:460] Webserver started at http://127.0.0.1:36149/ using document root <none> and password file <none>
I20260808 11:42:46.892460 24859 fs_manager.cc:359] Metadata directory not provided
I20260808 11:42:46.892584 24859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:42:46.892871 24859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:42:46.895848 24859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "5d4ddc5848ad4d30bbee5b80276f9307"
format_stamp: "Formatted at 2026-08-08 11:42:46 on dist-test-slave-tc2s"
server_key: "43e7751f63b022b5286205b5bb7568cb"
server_key_iv: "c654c79d658130c0de69a7eb40db9f04"
server_key_version: "encryptionkey@0"
I20260808 11:42:46.896615 24859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "5d4ddc5848ad4d30bbee5b80276f9307"
format_stamp: "Formatted at 2026-08-08 11:42:46 on dist-test-slave-tc2s"
server_key: "43e7751f63b022b5286205b5bb7568cb"
server_key_iv: "c654c79d658130c0de69a7eb40db9f04"
server_key_version: "encryptionkey@0"
I20260808 11:42:46.901607 24859 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 11:42:46.904922 24893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:46.906208 24859 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 11:42:46.906394 24859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "5d4ddc5848ad4d30bbee5b80276f9307"
format_stamp: "Formatted at 2026-08-08 11:42:46 on dist-test-slave-tc2s"
server_key: "43e7751f63b022b5286205b5bb7568cb"
server_key_iv: "c654c79d658130c0de69a7eb40db9f04"
server_key_version: "encryptionkey@0"
I20260808 11:42:46.906574 24859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:42:46.922241 24859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:42:46.926131 24859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:42:46.926393 24859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:42:46.943984 24859 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41705
I20260808 11:42:46.944015 24982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41705 every 8 connection(s)
I20260808 11:42:46.945312 24859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:42:46.948881 24984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:42:46.952143 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 24859
I20260808 11:42:46.952320 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:42:46.952880 24881 external_mini_cluster.cc:1428] Setting key 69cd5f35499a089f02482f9f915f42e1
I20260808 11:42:46.960724 24984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d4ddc5848ad4d30bbee5b80276f9307: Bootstrap starting.
I20260808 11:42:46.964148 24984 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d4ddc5848ad4d30bbee5b80276f9307: Neither blocks nor log segments found. Creating new log.
I20260808 11:42:46.965251 24984 log.cc:824] T 00000000000000000000000000000000 P 5d4ddc5848ad4d30bbee5b80276f9307: Log is configured to *not* fsync() on all Append() calls
I20260808 11:42:46.965708 24990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:46.954654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34209 (local address 127.0.0.1:41705)
0808 11:42:46.955361 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:46.955372 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:46.955414 (+ 42us) server_negotiation.cc:407] Connection header received
0808 11:42:46.957047 (+ 1633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:46.957069 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:46.957445 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:46.957759 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:46.958369 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:46.959679 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:46.961449 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:46.962149 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:46.962280 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:46.963534 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:46.963542 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:46.963555 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:42:46.964035 (+ 480us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:46.964112 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:42:46.964139 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:46.964273 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:46.964560 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 11:42:46.964733 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":321,"threads_started":1}
I20260808 11:42:46.968279 24984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d4ddc5848ad4d30bbee5b80276f9307: No bootstrap required, opened a new log
I20260808 11:42:46.969509 24881 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:42:46.976593 24984 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d4ddc5848ad4d30bbee5b80276f9307 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d4ddc5848ad4d30bbee5b80276f9307" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41705 } }
I20260808 11:42:46.976935 24984 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d4ddc5848ad4d30bbee5b80276f9307 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:42:46.977025 24984 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d4ddc5848ad4d30bbee5b80276f9307 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4ddc5848ad4d30bbee5b80276f9307, State: Initialized, Role: FOLLOWER
I20260808 11:42:46.977594 24984 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d4ddc5848ad4d30bbee5b80276f9307 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d4ddc5848ad4d30bbee5b80276f9307" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41705 } }
I20260808 11:42:46.977838 24984 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d4ddc5848ad4d30bbee5b80276f9307 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:42:46.977999 24984 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d4ddc5848ad4d30bbee5b80276f9307 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:42:46.978170 24984 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d4ddc5848ad4d30bbee5b80276f9307 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:42:46.979848 24990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:46.973678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38629 (local address 127.0.0.1:41705)
0808 11:42:46.973858 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:46.973867 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:46.974157 (+ 290us) server_negotiation.cc:407] Connection header received
0808 11:42:46.974303 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:46.974308 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:46.974383 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:46.974490 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:46.974920 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:46.976067 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:46.977413 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:46.977870 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:46.978063 (+ 193us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:46.979128 (+ 1065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:46.979133 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:46.979137 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:42:46.979160 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:46.979218 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:42:46.979234 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:46.979331 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:46.979545 (+ 214us) server_negotiation.cc:299] Negotiation successful
0808 11:42:46.979599 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 11:42:46.980732 24984 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d4ddc5848ad4d30bbee5b80276f9307 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d4ddc5848ad4d30bbee5b80276f9307" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41705 } }
I20260808 11:42:46.981303 24984 leader_election.cc:302] T 00000000000000000000000000000000 P 5d4ddc5848ad4d30bbee5b80276f9307 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d4ddc5848ad4d30bbee5b80276f9307; no voters:
I20260808 11:42:46.981871 24984 leader_election.cc:288] T 00000000000000000000000000000000 P 5d4ddc5848ad4d30bbee5b80276f9307 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:42:46.982034 25006 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d4ddc5848ad4d30bbee5b80276f9307 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:42:46.982658 25006 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d4ddc5848ad4d30bbee5b80276f9307 [term 1 LEADER]: Becoming Leader. State: Replica: 5d4ddc5848ad4d30bbee5b80276f9307, State: Running, Role: LEADER
I20260808 11:42:46.983186 25006 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d4ddc5848ad4d30bbee5b80276f9307 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d4ddc5848ad4d30bbee5b80276f9307" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41705 } }
I20260808 11:42:46.983872 24984 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d4ddc5848ad4d30bbee5b80276f9307 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:42:46.986203 25008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d4ddc5848ad4d30bbee5b80276f9307 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d4ddc5848ad4d30bbee5b80276f9307. Latest consensus state: current_term: 1 leader_uuid: "5d4ddc5848ad4d30bbee5b80276f9307" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d4ddc5848ad4d30bbee5b80276f9307" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41705 } } }
I20260808 11:42:46.986598 25008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d4ddc5848ad4d30bbee5b80276f9307 [sys.catalog]: This master's current role is: LEADER
I20260808 11:42:46.987470 25007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d4ddc5848ad4d30bbee5b80276f9307 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d4ddc5848ad4d30bbee5b80276f9307" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d4ddc5848ad4d30bbee5b80276f9307" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41705 } } }
I20260808 11:42:46.987793 25007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d4ddc5848ad4d30bbee5b80276f9307 [sys.catalog]: This master's current role is: LEADER
I20260808 11:42:46.995047 25017 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:42:46.999047 25017 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:42:47.006619 25017 catalog_manager.cc:1347] Generated new cluster ID: aa8a22b3505643b094af000518f377c4
I20260808 11:42:47.006737 25017 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:42:47.026991 25017 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:42:47.028457 25017 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:42:47.039968 25017 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d4ddc5848ad4d30bbee5b80276f9307: Generated new TSK 0
I20260808 11:42:47.040728 25017 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:42:47.055713 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 24859
W20260808 11:42:47.056911 24971 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34209
W20260808 11:42:47.057626 24881 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 24859: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:42:47.057816 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 24859
W20260808 11:42:47.080365 24831 connection.cc:535] client connection to 127.0.0.1:41705 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41705): Connection reset by peer (error 104)
2026-08-08T11:42:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2063 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:42:47 dist-test-slave-tc2s krb5kdc[25043](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:42:47 dist-test-slave-tc2s krb5kdc[25043](info): set up 2 sockets
Aug 08 11:42:47 dist-test-slave-tc2s krb5kdc[25043](info): commencing operation
krb5kdc: starting...
W20260808 11:42:48.296298 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:42:48 dist-test-slave-tc2s krb5kdc[25043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:42:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:42:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 11:42:48.580856 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37659
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37659
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:42:48.908596 25065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:42:48.908857 25065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:42:48.908987 25065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:42:48.915833 25065 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:42:48.915928 25065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:42:48.915987 25065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:42:48.916070 25065 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:42:48.916127 25065 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:42:48.925381 25065 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37659
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:42:48.926543 25065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:42:48.927768 25065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:42:48.938839 25072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:48.938772 25073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:48.938804 25075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:42:48.939599 25065 server_base.cc:1034] running on GCE node
I20260808 11:42:48.940375 25065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:42:48.941613 25065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:42:48.942790 25065 hybrid_clock.cc:648] HybridClock initialized: now 1786189368942745 us; error 42 us; skew 500 ppm
I20260808 11:42:48.943109 25065 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:42:48 dist-test-slave-tc2s krb5kdc[25043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189368, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:42:48.946043 25065 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 11:42:48.947320 25065 webserver.cc:460] Webserver started at http://127.0.0.1:46615/ using document root <none> and password file <none>
I20260808 11:42:48.947911 25065 fs_manager.cc:359] Metadata directory not provided
I20260808 11:42:48.948009 25065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:42:48.948254 25065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:42:48.951045 25065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "89cff64a76dd4426a74bac20ff631013"
format_stamp: "Formatted at 2026-08-08 11:42:48 on dist-test-slave-tc2s"
server_key: "7c4f5e5cc021a8b7b4083c4394621314"
server_key_iv: "33c03af1423c625396e5fcd697479575"
server_key_version: "encryptionkey@0"
I20260808 11:42:48.951742 25065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "89cff64a76dd4426a74bac20ff631013"
format_stamp: "Formatted at 2026-08-08 11:42:48 on dist-test-slave-tc2s"
server_key: "7c4f5e5cc021a8b7b4083c4394621314"
server_key_iv: "33c03af1423c625396e5fcd697479575"
server_key_version: "encryptionkey@0"
I20260808 11:42:48.956465 25065 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 11:42:48.959640 25086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:48.960834 25065 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 11:42:48.960989 25065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "89cff64a76dd4426a74bac20ff631013"
format_stamp: "Formatted at 2026-08-08 11:42:48 on dist-test-slave-tc2s"
server_key: "7c4f5e5cc021a8b7b4083c4394621314"
server_key_iv: "33c03af1423c625396e5fcd697479575"
server_key_version: "encryptionkey@0"
I20260808 11:42:48.961170 25065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:42:48.966606 25065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:42:48.970126 25065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:42:48.970383 25065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:42:48.987100 25065 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37659
I20260808 11:42:48.987156 25182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37659 every 8 connection(s)
I20260808 11:42:48.988384 25065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:42:48.991847 25184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:42:48.997223 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 25065
I20260808 11:42:48.997399 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:42:48.997833 24881 external_mini_cluster.cc:1428] Setting key 56657476ea0b829d9e221669be48393e
I20260808 11:42:49.002982 25184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89cff64a76dd4426a74bac20ff631013: Bootstrap starting.
I20260808 11:42:49.006436 25188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:48.999608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54301 (local address 127.0.0.1:37659)
0808 11:42:49.000518 (+ 910us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:49.000532 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:49.000587 (+ 55us) server_negotiation.cc:407] Connection header received
0808 11:42:49.002830 (+ 2243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:49.002857 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:49.003258 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:49.003665 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:49.004605 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:49.004617 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:49.004634 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:42:49.004656 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:49.004732 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:42:49.004760 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:49.004969 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:49.005095 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 11:42:49.005281 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":370,"threads_started":1}
I20260808 11:42:49.006803 25184 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89cff64a76dd4426a74bac20ff631013: Neither blocks nor log segments found. Creating new log.
I20260808 11:42:49.007791 25184 log.cc:824] T 00000000000000000000000000000000 P 89cff64a76dd4426a74bac20ff631013: Log is configured to *not* fsync() on all Append() calls
I20260808 11:42:49.009909 24881 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:42:49.010578 25184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89cff64a76dd4426a74bac20ff631013: No bootstrap required, opened a new log
I20260808 11:42:49.016270 25188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:49.014177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59397 (local address 127.0.0.1:37659)
0808 11:42:49.014345 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:49.014351 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:49.014576 (+ 225us) server_negotiation.cc:407] Connection header received
0808 11:42:49.014697 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:49.014702 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:49.014759 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:49.014865 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:49.015675 (+ 810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:49.015680 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:49.015685 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:42:49.015707 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:49.015757 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:42:49.015776 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:49.015903 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:49.015945 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 11:42:49.015993 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 11:42:49.018464 25184 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89cff64a76dd4426a74bac20ff631013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89cff64a76dd4426a74bac20ff631013" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37659 } }
I20260808 11:42:49.018786 25184 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89cff64a76dd4426a74bac20ff631013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:42:49.018857 25184 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89cff64a76dd4426a74bac20ff631013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89cff64a76dd4426a74bac20ff631013, State: Initialized, Role: FOLLOWER
I20260808 11:42:49.019301 25184 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89cff64a76dd4426a74bac20ff631013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89cff64a76dd4426a74bac20ff631013" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37659 } }
I20260808 11:42:49.019515 25184 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89cff64a76dd4426a74bac20ff631013 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:42:49.019605 25184 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89cff64a76dd4426a74bac20ff631013 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:42:49.019729 25184 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89cff64a76dd4426a74bac20ff631013 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:42:49.021744 25184 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89cff64a76dd4426a74bac20ff631013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89cff64a76dd4426a74bac20ff631013" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37659 } }
I20260808 11:42:49.022229 25184 leader_election.cc:302] T 00000000000000000000000000000000 P 89cff64a76dd4426a74bac20ff631013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89cff64a76dd4426a74bac20ff631013; no voters:
I20260808 11:42:49.022745 25184 leader_election.cc:288] T 00000000000000000000000000000000 P 89cff64a76dd4426a74bac20ff631013 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:42:49.022866 25196 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89cff64a76dd4426a74bac20ff631013 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:42:49.023478 25196 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89cff64a76dd4426a74bac20ff631013 [term 1 LEADER]: Becoming Leader. State: Replica: 89cff64a76dd4426a74bac20ff631013, State: Running, Role: LEADER
I20260808 11:42:49.023972 25196 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89cff64a76dd4426a74bac20ff631013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89cff64a76dd4426a74bac20ff631013" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37659 } }
I20260808 11:42:49.024623 25184 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89cff64a76dd4426a74bac20ff631013 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:42:49.026532 25197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89cff64a76dd4426a74bac20ff631013 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89cff64a76dd4426a74bac20ff631013" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89cff64a76dd4426a74bac20ff631013" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37659 } } }
I20260808 11:42:49.026984 25197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89cff64a76dd4426a74bac20ff631013 [sys.catalog]: This master's current role is: LEADER
I20260808 11:42:49.028013 25198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89cff64a76dd4426a74bac20ff631013 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89cff64a76dd4426a74bac20ff631013. Latest consensus state: current_term: 1 leader_uuid: "89cff64a76dd4426a74bac20ff631013" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89cff64a76dd4426a74bac20ff631013" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37659 } } }
I20260808 11:42:49.028286 25198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89cff64a76dd4426a74bac20ff631013 [sys.catalog]: This master's current role is: LEADER
I20260808 11:42:49.030247 25206 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:42:49.035431 25206 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:42:49.044422 25206 catalog_manager.cc:1347] Generated new cluster ID: 1f9d0d881d064598ad4efd30000cdf36
I20260808 11:42:49.044518 25206 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:42:49.056726 25206 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:42:49.057082 25206 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:42:49.065433 25206 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89cff64a76dd4426a74bac20ff631013: Generated new TSK 0
I20260808 11:42:49.066341 25206 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:42:49.071607 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 25065
W20260808 11:42:49.072599 25172 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:54301
W20260808 11:42:49.073204 24881 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 25065: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:42:49.073356 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 25065
2026-08-08T11:42:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2018 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:42:49 dist-test-slave-tc2s krb5kdc[25233](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:42:49 dist-test-slave-tc2s krb5kdc[25233](info): set up 2 sockets
Aug 08 11:42:49 dist-test-slave-tc2s krb5kdc[25233](info): commencing operation
krb5kdc: starting...
W20260808 11:42:50.293395 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:42:50 dist-test-slave-tc2s krb5kdc[25233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:42:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:42:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 11:42:50.583141 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38299
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38299
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:42:50.914008 25251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:42:50.914276 25251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:42:50.914402 25251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:42:50.921337 25251 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:42:50.921434 25251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:42:50.921522 25251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:42:50.921603 25251 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:42:50.921659 25251 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:42:50.931032 25251 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38299
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:42:50.932179 25251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:42:50.933410 25251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:42:50.944736 25260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:50.944780 25265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:50.944928 25261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:42:50.945772 25251 server_base.cc:1034] running on GCE node
I20260808 11:42:50.946511 25251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:42:50.947775 25251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:42:50.948958 25251 hybrid_clock.cc:648] HybridClock initialized: now 1786189370948910 us; error 39 us; skew 500 ppm
I20260808 11:42:50.949273 25251 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:42:50 dist-test-slave-tc2s krb5kdc[25233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189370, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:42:50.957006 25251 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 11:42:50.958429 25251 webserver.cc:460] Webserver started at http://127.0.0.1:43199/ using document root <none> and password file <none>
I20260808 11:42:50.959085 25251 fs_manager.cc:359] Metadata directory not provided
I20260808 11:42:50.959204 25251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:42:50.959487 25251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:42:50.962406 25251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "53b8999b5f994b869c9ce5436aa7b0dd"
format_stamp: "Formatted at 2026-08-08 11:42:50 on dist-test-slave-tc2s"
server_key: "c8be9edefbe82f75dafc482743a08053"
server_key_iv: "9adfa27eff00877aef806bec4ab9b1f5"
server_key_version: "encryptionkey@0"
I20260808 11:42:50.963163 25251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "53b8999b5f994b869c9ce5436aa7b0dd"
format_stamp: "Formatted at 2026-08-08 11:42:50 on dist-test-slave-tc2s"
server_key: "c8be9edefbe82f75dafc482743a08053"
server_key_iv: "9adfa27eff00877aef806bec4ab9b1f5"
server_key_version: "encryptionkey@0"
I20260808 11:42:50.968060 25251 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 11:42:50.971403 25277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:50.972791 25251 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 11:42:50.972945 25251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "53b8999b5f994b869c9ce5436aa7b0dd"
format_stamp: "Formatted at 2026-08-08 11:42:50 on dist-test-slave-tc2s"
server_key: "c8be9edefbe82f75dafc482743a08053"
server_key_iv: "9adfa27eff00877aef806bec4ab9b1f5"
server_key_version: "encryptionkey@0"
I20260808 11:42:50.973117 25251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:42:50.990381 25251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:42:50.993953 25251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:42:50.994194 25251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:42:51.010502 25251 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38299
I20260808 11:42:51.010542 25356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38299 every 8 connection(s)
I20260808 11:42:51.011814 25251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:42:51.015450 25358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:42:51.021835 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 25251
I20260808 11:42:51.022058 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:42:51.022495 24881 external_mini_cluster.cc:1428] Setting key e294b4f4d1c2055ff0d6620d698aaa79
I20260808 11:42:51.026149 25358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53b8999b5f994b869c9ce5436aa7b0dd: Bootstrap starting.
I20260808 11:42:51.029799 25358 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53b8999b5f994b869c9ce5436aa7b0dd: Neither blocks nor log segments found. Creating new log.
I20260808 11:42:51.031199 25358 log.cc:824] T 00000000000000000000000000000000 P 53b8999b5f994b869c9ce5436aa7b0dd: Log is configured to *not* fsync() on all Append() calls
I20260808 11:42:51.034289 25358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53b8999b5f994b869c9ce5436aa7b0dd: No bootstrap required, opened a new log
Aug 08 11:42:51 dist-test-slave-tc2s krb5kdc[25233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 11:42:51.042784 25358 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53b8999b5f994b869c9ce5436aa7b0dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b8999b5f994b869c9ce5436aa7b0dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38299 } }
I20260808 11:42:51.043138 25358 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53b8999b5f994b869c9ce5436aa7b0dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:42:51.043284 25358 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53b8999b5f994b869c9ce5436aa7b0dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53b8999b5f994b869c9ce5436aa7b0dd, State: Initialized, Role: FOLLOWER
I20260808 11:42:51.043862 25358 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53b8999b5f994b869c9ce5436aa7b0dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b8999b5f994b869c9ce5436aa7b0dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38299 } }
I20260808 11:42:51.044124 25358 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53b8999b5f994b869c9ce5436aa7b0dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:42:51.044260 25358 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53b8999b5f994b869c9ce5436aa7b0dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:42:51.044427 25358 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53b8999b5f994b869c9ce5436aa7b0dd [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:42:51.046494 25362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:51.024199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56753 (local address 127.0.0.1:38299)
0808 11:42:51.024974 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:51.024989 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:51.025038 (+ 49us) server_negotiation.cc:407] Connection header received
0808 11:42:51.027393 (+ 2355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:51.027421 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:51.027958 (+ 537us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:51.028392 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:51.029648 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:51.031670 (+ 2022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:51.033347 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:51.034077 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:51.034212 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:51.038878 (+ 4666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:51.038886 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:51.038900 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:42:51.038946 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:51.042293 (+ 3347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:51.042976 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:51.042982 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:51.042988 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:51.043223 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:51.043651 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:51.043657 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:51.043660 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:51.044260 (+ 600us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:51.044558 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:51.045035 (+ 477us) server_negotiation.cc:299] Negotiation successful
0808 11:42:51.045466 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":307,"threads_started":1}
I20260808 11:42:51.046931 25358 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53b8999b5f994b869c9ce5436aa7b0dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b8999b5f994b869c9ce5436aa7b0dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38299 } }
I20260808 11:42:51.047403 25358 leader_election.cc:302] T 00000000000000000000000000000000 P 53b8999b5f994b869c9ce5436aa7b0dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53b8999b5f994b869c9ce5436aa7b0dd; no voters:
I20260808 11:42:51.047972 25358 leader_election.cc:288] T 00000000000000000000000000000000 P 53b8999b5f994b869c9ce5436aa7b0dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:42:51.048064 25364 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53b8999b5f994b869c9ce5436aa7b0dd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:42:51.049592 25364 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53b8999b5f994b869c9ce5436aa7b0dd [term 1 LEADER]: Becoming Leader. State: Replica: 53b8999b5f994b869c9ce5436aa7b0dd, State: Running, Role: LEADER
I20260808 11:42:51.050179 25358 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53b8999b5f994b869c9ce5436aa7b0dd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:42:51.050204 25364 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53b8999b5f994b869c9ce5436aa7b0dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b8999b5f994b869c9ce5436aa7b0dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38299 } }
I20260808 11:42:51.050650 24881 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:42:51.055150 25366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53b8999b5f994b869c9ce5436aa7b0dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53b8999b5f994b869c9ce5436aa7b0dd. Latest consensus state: current_term: 1 leader_uuid: "53b8999b5f994b869c9ce5436aa7b0dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b8999b5f994b869c9ce5436aa7b0dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38299 } } }
I20260808 11:42:51.055522 25366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53b8999b5f994b869c9ce5436aa7b0dd [sys.catalog]: This master's current role is: LEADER
I20260808 11:42:51.056061 25365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53b8999b5f994b869c9ce5436aa7b0dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53b8999b5f994b869c9ce5436aa7b0dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b8999b5f994b869c9ce5436aa7b0dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38299 } } }
I20260808 11:42:51.056362 25365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53b8999b5f994b869c9ce5436aa7b0dd [sys.catalog]: This master's current role is: LEADER
I20260808 11:42:51.061959 25378 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:42:51.070698 25378 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:42:51.084973 25362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:51.068029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41909 (local address 127.0.0.1:38299)
0808 11:42:51.068220 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:51.068226 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:51.068446 (+ 220us) server_negotiation.cc:407] Connection header received
0808 11:42:51.068723 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:51.068729 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:51.068811 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:51.068904 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:51.070777 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:51.071892 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:51.073744 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:51.074482 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:51.074581 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:51.077795 (+ 3214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:51.077802 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:51.077807 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:42:51.077845 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:51.081606 (+ 3761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:51.082477 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:51.082482 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:51.082486 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:51.082632 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:51.083110 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:51.083115 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:51.083119 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:51.083540 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:51.083756 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:51.084378 (+ 622us) server_negotiation.cc:299] Negotiation successful
0808 11:42:51.084619 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 11:42:51.093390 25378 catalog_manager.cc:1347] Generated new cluster ID: fe19c3d5f9534dfa8fb0766f71e80658
I20260808 11:42:51.093505 25378 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:42:51.118412 25378 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:42:51.119837 25378 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:42:51.129628 25378 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53b8999b5f994b869c9ce5436aa7b0dd: Generated new TSK 0
I20260808 11:42:51.130733 25378 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:42:51.141481 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 25251
I20260808 11:42:51.143221 25351 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:42:51.233341 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 25251
2026-08-08T11:42:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2156 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:42:51 dist-test-slave-tc2s krb5kdc[25402](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:42:51 dist-test-slave-tc2s krb5kdc[25402](info): set up 2 sockets
Aug 08 11:42:51 dist-test-slave-tc2s krb5kdc[25402](info): commencing operation
krb5kdc: starting...
W20260808 11:42:52.457175 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:42:52 dist-test-slave-tc2s krb5kdc[25402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:42:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:42:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 11:42:52.746538 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35381
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:42:53.074241 25426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:42:53.074528 25426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:42:53.074667 25426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:42:53.081646 25426 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:42:53.081760 25426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:42:53.081838 25426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:42:53.081949 25426 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:42:53.082012 25426 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:42:53.091466 25426 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35381
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:42:53.092648 25426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:42:53.093935 25426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:42:53.105347 25432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:53.105312 25438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:53.105281 25434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:42:53.106272 25426 server_base.cc:1034] running on GCE node
I20260808 11:42:53.106989 25426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:42:53.108249 25426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:42:53.109447 25426 hybrid_clock.cc:648] HybridClock initialized: now 1786189373109401 us; error 33 us; skew 500 ppm
I20260808 11:42:53.109769 25426 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:42:53 dist-test-slave-tc2s krb5kdc[25402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189373, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:42:53.112717 25426 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 11:42:53.114194 25426 webserver.cc:460] Webserver started at http://127.0.0.1:39517/ using document root <none> and password file <none>
I20260808 11:42:53.114842 25426 fs_manager.cc:359] Metadata directory not provided
I20260808 11:42:53.114959 25426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:42:53.115242 25426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:42:53.118108 25426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "ed54dd8f8f144f6b939e224295f72df8"
format_stamp: "Formatted at 2026-08-08 11:42:53 on dist-test-slave-tc2s"
server_key: "5212113bfb345acc1d72566ccf80e33c"
server_key_iv: "8822157e88baa7c1b977b4aa2c72795a"
server_key_version: "encryptionkey@0"
I20260808 11:42:53.118850 25426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "ed54dd8f8f144f6b939e224295f72df8"
format_stamp: "Formatted at 2026-08-08 11:42:53 on dist-test-slave-tc2s"
server_key: "5212113bfb345acc1d72566ccf80e33c"
server_key_iv: "8822157e88baa7c1b977b4aa2c72795a"
server_key_version: "encryptionkey@0"
I20260808 11:42:53.123607 25426 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 11:42:53.126758 25449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:53.127941 25426 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 11:42:53.128088 25426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "ed54dd8f8f144f6b939e224295f72df8"
format_stamp: "Formatted at 2026-08-08 11:42:53 on dist-test-slave-tc2s"
server_key: "5212113bfb345acc1d72566ccf80e33c"
server_key_iv: "8822157e88baa7c1b977b4aa2c72795a"
server_key_version: "encryptionkey@0"
I20260808 11:42:53.128258 25426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:42:53.146930 25426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:42:53.150465 25426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:42:53.150710 25426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:42:53.167554 25426 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35381
I20260808 11:42:53.167577 25544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35381 every 8 connection(s)
I20260808 11:42:53.168758 25426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:42:53.172217 25545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:42:53.173249 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 25426
I20260808 11:42:53.173431 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:42:53.174019 24881 external_mini_cluster.cc:1428] Setting key 78383b11d11e70e637587c46e5aac916
I20260808 11:42:53.185117 25545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed54dd8f8f144f6b939e224295f72df8: Bootstrap starting.
I20260808 11:42:53.186589 25549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:53.175694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33133 (local address 127.0.0.1:35381)
0808 11:42:53.176407 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:53.176419 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:53.176459 (+ 40us) server_negotiation.cc:407] Connection header received
0808 11:42:53.178168 (+ 1709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:53.178190 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:53.178557 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:53.178864 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:53.179443 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:53.180681 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:53.182527 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:53.183177 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:53.183297 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:53.184448 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:53.184456 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:53.184468 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:42:53.184963 (+ 495us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:53.185036 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:42:53.185059 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:53.185152 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:53.185376 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 11:42:53.185623 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":319,"threads_started":1}
I20260808 11:42:53.188171 25545 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed54dd8f8f144f6b939e224295f72df8: Neither blocks nor log segments found. Creating new log.
I20260808 11:42:53.189412 25545 log.cc:824] T 00000000000000000000000000000000 P ed54dd8f8f144f6b939e224295f72df8: Log is configured to *not* fsync() on all Append() calls
I20260808 11:42:53.189965 24881 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:42:53.191942 25545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed54dd8f8f144f6b939e224295f72df8: No bootstrap required, opened a new log
I20260808 11:42:53.200012 25549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:53.193998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47883 (local address 127.0.0.1:35381)
0808 11:42:53.194172 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:53.194177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:53.194601 (+ 424us) server_negotiation.cc:407] Connection header received
0808 11:42:53.194744 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:53.194749 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:53.194834 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:53.194944 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:53.195377 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:53.196416 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:53.197606 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:53.198322 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:53.198440 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:53.199313 (+ 873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:53.199319 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:53.199324 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:42:53.199348 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:53.199390 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:42:53.199409 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:53.199503 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:53.199607 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 11:42:53.199691 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 11:42:53.200692 25545 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed54dd8f8f144f6b939e224295f72df8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed54dd8f8f144f6b939e224295f72df8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35381 } }
I20260808 11:42:53.201038 25545 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed54dd8f8f144f6b939e224295f72df8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:42:53.201133 25545 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed54dd8f8f144f6b939e224295f72df8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed54dd8f8f144f6b939e224295f72df8, State: Initialized, Role: FOLLOWER
I20260808 11:42:53.201655 25545 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed54dd8f8f144f6b939e224295f72df8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed54dd8f8f144f6b939e224295f72df8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35381 } }
I20260808 11:42:53.201884 25545 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed54dd8f8f144f6b939e224295f72df8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:42:53.202066 25545 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed54dd8f8f144f6b939e224295f72df8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:42:53.202217 25545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed54dd8f8f144f6b939e224295f72df8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:42:53.204299 25545 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed54dd8f8f144f6b939e224295f72df8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed54dd8f8f144f6b939e224295f72df8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35381 } }
I20260808 11:42:53.204761 25545 leader_election.cc:302] T 00000000000000000000000000000000 P ed54dd8f8f144f6b939e224295f72df8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed54dd8f8f144f6b939e224295f72df8; no voters:
I20260808 11:42:53.205266 25545 leader_election.cc:288] T 00000000000000000000000000000000 P ed54dd8f8f144f6b939e224295f72df8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:42:53.205377 25559 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed54dd8f8f144f6b939e224295f72df8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:42:53.206444 25559 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed54dd8f8f144f6b939e224295f72df8 [term 1 LEADER]: Becoming Leader. State: Replica: ed54dd8f8f144f6b939e224295f72df8, State: Running, Role: LEADER
I20260808 11:42:53.206905 25559 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed54dd8f8f144f6b939e224295f72df8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed54dd8f8f144f6b939e224295f72df8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35381 } }
I20260808 11:42:53.207229 25545 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed54dd8f8f144f6b939e224295f72df8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:42:53.209316 25562 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed54dd8f8f144f6b939e224295f72df8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed54dd8f8f144f6b939e224295f72df8. Latest consensus state: current_term: 1 leader_uuid: "ed54dd8f8f144f6b939e224295f72df8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed54dd8f8f144f6b939e224295f72df8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35381 } } }
I20260808 11:42:53.209673 25562 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed54dd8f8f144f6b939e224295f72df8 [sys.catalog]: This master's current role is: LEADER
I20260808 11:42:53.210234 25560 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed54dd8f8f144f6b939e224295f72df8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed54dd8f8f144f6b939e224295f72df8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed54dd8f8f144f6b939e224295f72df8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35381 } } }
I20260808 11:42:53.210561 25560 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed54dd8f8f144f6b939e224295f72df8 [sys.catalog]: This master's current role is: LEADER
I20260808 11:42:53.221006 25567 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:42:53.224916 25567 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:42:53.232685 25567 catalog_manager.cc:1347] Generated new cluster ID: 7e0f9b84bac24292a821471e76a258fd
I20260808 11:42:53.232782 25567 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:42:53.241905 25567 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:42:53.243052 25567 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:42:53.263435 25567 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed54dd8f8f144f6b939e224295f72df8: Generated new TSK 0
I20260808 11:42:53.264317 25567 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:42:53.280696 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 25426
W20260808 11:42:53.281834 25538 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33133
W20260808 11:42:53.282509 24881 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 25426: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:42:53.282662 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 25426
2026-08-08T11:42:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2051 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:42:53 dist-test-slave-tc2s krb5kdc[25594](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:42:53 dist-test-slave-tc2s krb5kdc[25594](info): set up 2 sockets
Aug 08 11:42:53 dist-test-slave-tc2s krb5kdc[25594](info): commencing operation
krb5kdc: starting...
W20260808 11:42:54.500687 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:42:54 dist-test-slave-tc2s krb5kdc[25594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:42:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:42:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 11:42:54.804682 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42293
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:42:55.139104 25617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:42:55.139398 25617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:42:55.139537 25617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:42:55.147373 25617 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:42:55.147503 25617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:42:55.147581 25617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:42:55.147644 25617 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:42:55.147692 25617 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:42:55.157522 25617 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42293
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:42:55.158708 25617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:42:55.159889 25617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:42:55.170800 25631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:55.170886 25625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:55.170857 25627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:42:55.171983 25617 server_base.cc:1034] running on GCE node
I20260808 11:42:55.172716 25617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:42:55.173987 25617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:42:55.175212 25617 hybrid_clock.cc:648] HybridClock initialized: now 1786189375175163 us; error 39 us; skew 500 ppm
I20260808 11:42:55.175554 25617 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:42:55 dist-test-slave-tc2s krb5kdc[25594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189375, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:42:55.178511 25617 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 11:42:55.179889 25617 webserver.cc:460] Webserver started at http://127.0.0.1:39827/ using document root <none> and password file <none>
I20260808 11:42:55.180531 25617 fs_manager.cc:359] Metadata directory not provided
I20260808 11:42:55.180648 25617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:42:55.180938 25617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:42:55.183960 25617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "f74335a429b54962bb9d3ae0e4980f44"
format_stamp: "Formatted at 2026-08-08 11:42:55 on dist-test-slave-tc2s"
server_key: "1c3d1de14184b4918425bfbbb7fb48ab"
server_key_iv: "f8311d017910864a81606383a056942e"
server_key_version: "encryptionkey@0"
I20260808 11:42:55.184711 25617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "f74335a429b54962bb9d3ae0e4980f44"
format_stamp: "Formatted at 2026-08-08 11:42:55 on dist-test-slave-tc2s"
server_key: "1c3d1de14184b4918425bfbbb7fb48ab"
server_key_iv: "f8311d017910864a81606383a056942e"
server_key_version: "encryptionkey@0"
I20260808 11:42:55.189481 25617 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260808 11:42:55.192690 25645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:55.193883 25617 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 11:42:55.194075 25617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "f74335a429b54962bb9d3ae0e4980f44"
format_stamp: "Formatted at 2026-08-08 11:42:55 on dist-test-slave-tc2s"
server_key: "1c3d1de14184b4918425bfbbb7fb48ab"
server_key_iv: "f8311d017910864a81606383a056942e"
server_key_version: "encryptionkey@0"
I20260808 11:42:55.194245 25617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:42:55.199643 25617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:42:55.203259 25617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:42:55.203528 25617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:42:55.219972 25617 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42293
I20260808 11:42:55.220024 25711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42293 every 8 connection(s)
I20260808 11:42:55.221232 25617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:42:55.223412 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 25617
I20260808 11:42:55.223608 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:42:55.224190 24881 external_mini_cluster.cc:1428] Setting key 361737cb6bae9ebbae0f95919dd16281
I20260808 11:42:55.225221 25712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:42:55.231192 25719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:55.225844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41353 (local address 127.0.0.1:42293)
0808 11:42:55.226597 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:55.226607 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:55.226645 (+ 38us) server_negotiation.cc:407] Connection header received
0808 11:42:55.228119 (+ 1474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:55.228140 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:55.228459 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:55.228763 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:55.229755 (+ 992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:55.229761 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:55.229777 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:42:55.229790 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:55.229858 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:42:55.229882 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:55.230075 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:55.230172 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 11:42:55.230339 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":307,"threads_started":1}
I20260808 11:42:55.234453 24881 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:42:55.238076 25712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f74335a429b54962bb9d3ae0e4980f44: Bootstrap starting.
I20260808 11:42:55.241732 25719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:55.239580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37929 (local address 127.0.0.1:42293)
0808 11:42:55.239748 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:55.239753 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:55.240051 (+ 298us) server_negotiation.cc:407] Connection header received
0808 11:42:55.240200 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:55.240203 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:55.240244 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:55.240323 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:55.241181 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:55.241184 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:55.241187 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:42:55.241201 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:55.241234 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:42:55.241248 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:55.241323 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:55.241444 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 11:42:55.241482 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 11:42:55.241739 25712 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f74335a429b54962bb9d3ae0e4980f44: Neither blocks nor log segments found. Creating new log.
I20260808 11:42:55.242856 25712 log.cc:824] T 00000000000000000000000000000000 P f74335a429b54962bb9d3ae0e4980f44: Log is configured to *not* fsync() on all Append() calls
I20260808 11:42:55.245201 25712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f74335a429b54962bb9d3ae0e4980f44: No bootstrap required, opened a new log
I20260808 11:42:55.252087 25712 raft_consensus.cc:348] T 00000000000000000000000000000000 P f74335a429b54962bb9d3ae0e4980f44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f74335a429b54962bb9d3ae0e4980f44" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42293 } }
I20260808 11:42:55.252408 25712 raft_consensus.cc:374] T 00000000000000000000000000000000 P f74335a429b54962bb9d3ae0e4980f44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:42:55.252493 25712 raft_consensus.cc:729] T 00000000000000000000000000000000 P f74335a429b54962bb9d3ae0e4980f44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f74335a429b54962bb9d3ae0e4980f44, State: Initialized, Role: FOLLOWER
I20260808 11:42:55.253002 25712 consensus_queue.cc:260] T 00000000000000000000000000000000 P f74335a429b54962bb9d3ae0e4980f44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f74335a429b54962bb9d3ae0e4980f44" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42293 } }
I20260808 11:42:55.253209 25712 raft_consensus.cc:388] T 00000000000000000000000000000000 P f74335a429b54962bb9d3ae0e4980f44 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:42:55.253330 25712 raft_consensus.cc:482] T 00000000000000000000000000000000 P f74335a429b54962bb9d3ae0e4980f44 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:42:55.253481 25712 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f74335a429b54962bb9d3ae0e4980f44 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:42:55.255561 25712 raft_consensus.cc:504] T 00000000000000000000000000000000 P f74335a429b54962bb9d3ae0e4980f44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f74335a429b54962bb9d3ae0e4980f44" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42293 } }
I20260808 11:42:55.256014 25712 leader_election.cc:302] T 00000000000000000000000000000000 P f74335a429b54962bb9d3ae0e4980f44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f74335a429b54962bb9d3ae0e4980f44; no voters:
I20260808 11:42:55.256521 25712 leader_election.cc:288] T 00000000000000000000000000000000 P f74335a429b54962bb9d3ae0e4980f44 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:42:55.256631 25730 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f74335a429b54962bb9d3ae0e4980f44 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:42:55.257225 25730 raft_consensus.cc:686] T 00000000000000000000000000000000 P f74335a429b54962bb9d3ae0e4980f44 [term 1 LEADER]: Becoming Leader. State: Replica: f74335a429b54962bb9d3ae0e4980f44, State: Running, Role: LEADER
I20260808 11:42:55.257776 25730 consensus_queue.cc:237] T 00000000000000000000000000000000 P f74335a429b54962bb9d3ae0e4980f44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f74335a429b54962bb9d3ae0e4980f44" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42293 } }
I20260808 11:42:55.258435 25712 sys_catalog.cc:564] T 00000000000000000000000000000000 P f74335a429b54962bb9d3ae0e4980f44 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:42:55.260375 25731 sys_catalog.cc:455] T 00000000000000000000000000000000 P f74335a429b54962bb9d3ae0e4980f44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f74335a429b54962bb9d3ae0e4980f44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f74335a429b54962bb9d3ae0e4980f44" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42293 } } }
I20260808 11:42:55.260840 25731 sys_catalog.cc:458] T 00000000000000000000000000000000 P f74335a429b54962bb9d3ae0e4980f44 [sys.catalog]: This master's current role is: LEADER
I20260808 11:42:55.260396 25734 sys_catalog.cc:455] T 00000000000000000000000000000000 P f74335a429b54962bb9d3ae0e4980f44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f74335a429b54962bb9d3ae0e4980f44. Latest consensus state: current_term: 1 leader_uuid: "f74335a429b54962bb9d3ae0e4980f44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f74335a429b54962bb9d3ae0e4980f44" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42293 } } }
I20260808 11:42:55.265959 25734 sys_catalog.cc:458] T 00000000000000000000000000000000 P f74335a429b54962bb9d3ae0e4980f44 [sys.catalog]: This master's current role is: LEADER
I20260808 11:42:55.267738 25741 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:42:55.272938 25741 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:42:55.280891 25741 catalog_manager.cc:1347] Generated new cluster ID: dd358a133f4742079c0e5ed761f12adb
I20260808 11:42:55.281059 25741 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:42:55.300750 25741 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:42:55.301142 25741 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:42:55.311038 25741 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f74335a429b54962bb9d3ae0e4980f44: Generated new TSK 0
I20260808 11:42:55.311765 25741 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:42:55.317911 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 25617
W20260808 11:42:55.318934 25706 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:41353
W20260808 11:42:55.319489 24881 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 25617: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:42:55.319615 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 25617
2026-08-08T11:42:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2035 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:42:55 dist-test-slave-tc2s krb5kdc[25769](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:42:55 dist-test-slave-tc2s krb5kdc[25769](info): set up 2 sockets
Aug 08 11:42:55 dist-test-slave-tc2s krb5kdc[25769](info): commencing operation
krb5kdc: starting...
W20260808 11:42:56.542775 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.001s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:42:56 dist-test-slave-tc2s krb5kdc[25769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:42:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:42:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 11:42:56.835191 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34593
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34593
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:42:57.166505 25784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:42:57.166788 25784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:42:57.166936 25784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:42:57.173931 25784 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:42:57.174043 25784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:42:57.174122 25784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:42:57.174212 25784 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:42:57.174276 25784 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:42:57.183693 25784 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34593
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:42:57.184885 25784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:42:57.186172 25784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:42:57.198853 25793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:57.198873 25792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:57.200503 25797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:42:57.200690 25784 server_base.cc:1034] running on GCE node
I20260808 11:42:57.201629 25784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:42:57.204108 25784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:42:57.205330 25784 hybrid_clock.cc:648] HybridClock initialized: now 1786189377205261 us; error 52 us; skew 500 ppm
I20260808 11:42:57.205731 25784 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:42:57 dist-test-slave-tc2s krb5kdc[25769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189377, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:42:57.209195 25784 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 11:42:57.210928 25784 webserver.cc:460] Webserver started at http://127.0.0.1:34133/ using document root <none> and password file <none>
I20260808 11:42:57.211694 25784 fs_manager.cc:359] Metadata directory not provided
I20260808 11:42:57.211812 25784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:42:57.212071 25784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:42:57.215924 25784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "70ec5de5460e4b3bbb60f240f2fcd163"
format_stamp: "Formatted at 2026-08-08 11:42:57 on dist-test-slave-tc2s"
server_key: "1a1a302a9e84fafe270bb37eaeded86d"
server_key_iv: "9305ab1edd0f5455891285838e028646"
server_key_version: "encryptionkey@0"
I20260808 11:42:57.216935 25784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "70ec5de5460e4b3bbb60f240f2fcd163"
format_stamp: "Formatted at 2026-08-08 11:42:57 on dist-test-slave-tc2s"
server_key: "1a1a302a9e84fafe270bb37eaeded86d"
server_key_iv: "9305ab1edd0f5455891285838e028646"
server_key_version: "encryptionkey@0"
I20260808 11:42:57.224822 25784 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260808 11:42:57.229161 25807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:57.230741 25784 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260808 11:42:57.230921 25784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "70ec5de5460e4b3bbb60f240f2fcd163"
format_stamp: "Formatted at 2026-08-08 11:42:57 on dist-test-slave-tc2s"
server_key: "1a1a302a9e84fafe270bb37eaeded86d"
server_key_iv: "9305ab1edd0f5455891285838e028646"
server_key_version: "encryptionkey@0"
I20260808 11:42:57.231099 25784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:42:57.251151 25784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:42:57.255093 25784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:42:57.255333 25784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:42:57.271643 25784 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34593
I20260808 11:42:57.271677 25899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34593 every 8 connection(s)
I20260808 11:42:57.272891 25784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:42:57.275204 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 25784
I20260808 11:42:57.275398 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:42:57.275934 24881 external_mini_cluster.cc:1428] Setting key 30301a00b4aed0d40d21995484f4f247
I20260808 11:42:57.276881 25900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 11:42:57 dist-test-slave-tc2s krb5kdc[25769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 11:42:57.290432 25900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70ec5de5460e4b3bbb60f240f2fcd163: Bootstrap starting.
I20260808 11:42:57.294183 25900 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70ec5de5460e4b3bbb60f240f2fcd163: Neither blocks nor log segments found. Creating new log.
I20260808 11:42:57.295346 25900 log.cc:824] T 00000000000000000000000000000000 P 70ec5de5460e4b3bbb60f240f2fcd163: Log is configured to *not* fsync() on all Append() calls
I20260808 11:42:57.298527 25900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70ec5de5460e4b3bbb60f240f2fcd163: No bootstrap required, opened a new log
I20260808 11:42:57.299005 25903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:57.277779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.137:52619 (local address 127.0.0.1:34593)
0808 11:42:57.278546 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:57.278561 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:57.278609 (+ 48us) server_negotiation.cc:407] Connection header received
0808 11:42:57.280381 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:57.280407 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:57.280898 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:57.281303 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:57.282609 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:57.284629 (+ 2020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:57.285834 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:57.286855 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:57.287027 (+ 172us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:57.291043 (+ 4016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:57.291053 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:57.291070 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:42:57.291118 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:57.294772 (+ 3654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:57.295480 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:57.295488 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:57.295495 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:57.295692 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:57.296165 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:57.296168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:57.296170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:57.296711 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:57.297013 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:57.297400 (+ 387us) server_negotiation.cc:299] Negotiation successful
0808 11:42:57.297704 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":294,"threads_started":1}
I20260808 11:42:57.302515 24881 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:42:57.306183 25900 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70ec5de5460e4b3bbb60f240f2fcd163 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70ec5de5460e4b3bbb60f240f2fcd163" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34593 } }
I20260808 11:42:57.306655 25900 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70ec5de5460e4b3bbb60f240f2fcd163 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:42:57.306767 25900 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70ec5de5460e4b3bbb60f240f2fcd163 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70ec5de5460e4b3bbb60f240f2fcd163, State: Initialized, Role: FOLLOWER
I20260808 11:42:57.307380 25900 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70ec5de5460e4b3bbb60f240f2fcd163 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70ec5de5460e4b3bbb60f240f2fcd163" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34593 } }
I20260808 11:42:57.307649 25900 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70ec5de5460e4b3bbb60f240f2fcd163 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:42:57.307785 25900 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70ec5de5460e4b3bbb60f240f2fcd163 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:42:57.307937 25900 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70ec5de5460e4b3bbb60f240f2fcd163 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:42:57.310540 25900 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70ec5de5460e4b3bbb60f240f2fcd163 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70ec5de5460e4b3bbb60f240f2fcd163" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34593 } }
I20260808 11:42:57.310989 25900 leader_election.cc:302] T 00000000000000000000000000000000 P 70ec5de5460e4b3bbb60f240f2fcd163 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70ec5de5460e4b3bbb60f240f2fcd163; no voters:
I20260808 11:42:57.311580 25900 leader_election.cc:288] T 00000000000000000000000000000000 P 70ec5de5460e4b3bbb60f240f2fcd163 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:42:57.311683 25917 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70ec5de5460e4b3bbb60f240f2fcd163 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:42:57.312808 25917 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70ec5de5460e4b3bbb60f240f2fcd163 [term 1 LEADER]: Becoming Leader. State: Replica: 70ec5de5460e4b3bbb60f240f2fcd163, State: Running, Role: LEADER
I20260808 11:42:57.313385 25917 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70ec5de5460e4b3bbb60f240f2fcd163 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70ec5de5460e4b3bbb60f240f2fcd163" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34593 } }
I20260808 11:42:57.313802 25900 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70ec5de5460e4b3bbb60f240f2fcd163 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:42:57.316269 25919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70ec5de5460e4b3bbb60f240f2fcd163 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70ec5de5460e4b3bbb60f240f2fcd163. Latest consensus state: current_term: 1 leader_uuid: "70ec5de5460e4b3bbb60f240f2fcd163" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70ec5de5460e4b3bbb60f240f2fcd163" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34593 } } }
I20260808 11:42:57.316674 25919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70ec5de5460e4b3bbb60f240f2fcd163 [sys.catalog]: This master's current role is: LEADER
I20260808 11:42:57.317234 25918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70ec5de5460e4b3bbb60f240f2fcd163 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70ec5de5460e4b3bbb60f240f2fcd163" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70ec5de5460e4b3bbb60f240f2fcd163" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34593 } } }
I20260808 11:42:57.317562 25918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70ec5de5460e4b3bbb60f240f2fcd163 [sys.catalog]: This master's current role is: LEADER
I20260808 11:42:57.323601 25903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:57.306613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.137:37607 (local address 127.0.0.1:34593)
0808 11:42:57.306750 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:57.306759 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:57.307259 (+ 500us) server_negotiation.cc:407] Connection header received
0808 11:42:57.307386 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:57.307391 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:57.307464 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:57.307561 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:57.308913 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:57.310046 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:57.311427 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:57.312028 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:57.312164 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:57.315021 (+ 2857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:57.315032 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:57.315036 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:42:57.315077 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:57.319237 (+ 4160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:57.320308 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:57.320314 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:57.320318 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:57.320517 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:42:57.321123 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:42:57.321129 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:42:57.321132 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:42:57.321520 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:57.322795 (+ 1275us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:57.323014 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 11:42:57.323193 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 11:42:57.330382 25926 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:42:57.335520 25926 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:42:57.344261 25926 catalog_manager.cc:1347] Generated new cluster ID: 346e937d377b46a4ae434bafe4c7a3fa
I20260808 11:42:57.344380 25926 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:42:57.362505 25926 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:42:57.364223 25926 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:42:57.371488 25926 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70ec5de5460e4b3bbb60f240f2fcd163: Generated new TSK 0
I20260808 11:42:57.372380 25926 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:42:57.377825 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 25784
I20260808 11:42:57.379305 25891 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:42:57.480630 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 25784
W20260808 11:42:57.503643 25765 connection.cc:535] client connection to 127.0.0.1:34593 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34593): Connection reset by peer (error 104)
2026-08-08T11:42:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2161 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:42:57 dist-test-slave-tc2s krb5kdc[25947](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:42:57 dist-test-slave-tc2s krb5kdc[25947](info): set up 2 sockets
Aug 08 11:42:57 dist-test-slave-tc2s krb5kdc[25947](info): commencing operation
krb5kdc: starting...
W20260808 11:42:58.708307 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.005s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:42:58 dist-test-slave-tc2s krb5kdc[25947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:42:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:42:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 11:42:59.004911 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44797
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44797
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:42:59.337513 25962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:42:59.337805 25962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:42:59.337950 25962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:42:59.345074 25962 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:42:59.345180 25962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:42:59.345248 25962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:42:59.345327 25962 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:42:59.345388 25962 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:42:59.354986 25962 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:42:59.356240 25962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:42:59.357580 25962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:42:59.368510 25968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:59.368575 25973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:42:59.368772 25969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:42:59.369876 25962 server_base.cc:1034] running on GCE node
I20260808 11:42:59.370632 25962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:42:59.371958 25962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:42:59.373152 25962 hybrid_clock.cc:648] HybridClock initialized: now 1786189379373108 us; error 34 us; skew 500 ppm
I20260808 11:42:59.373468 25962 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:42:59 dist-test-slave-tc2s krb5kdc[25947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189379, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:42:59.376555 25962 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 11:42:59.377847 25962 webserver.cc:460] Webserver started at http://127.0.0.1:45639/ using document root <none> and password file <none>
I20260808 11:42:59.378481 25962 fs_manager.cc:359] Metadata directory not provided
I20260808 11:42:59.378582 25962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:42:59.378822 25962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:42:59.381739 25962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "06ac98ac71b14709985f26622cc71589"
format_stamp: "Formatted at 2026-08-08 11:42:59 on dist-test-slave-tc2s"
server_key: "98fe09da5e7ff2f8f385f163e2d18ccb"
server_key_iv: "a5305162817785b0d35d07bcfc80426a"
server_key_version: "encryptionkey@0"
I20260808 11:42:59.382478 25962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "06ac98ac71b14709985f26622cc71589"
format_stamp: "Formatted at 2026-08-08 11:42:59 on dist-test-slave-tc2s"
server_key: "98fe09da5e7ff2f8f385f163e2d18ccb"
server_key_iv: "a5305162817785b0d35d07bcfc80426a"
server_key_version: "encryptionkey@0"
I20260808 11:42:59.387189 25962 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260808 11:42:59.390354 25984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:42:59.391566 25962 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260808 11:42:59.391700 25962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "06ac98ac71b14709985f26622cc71589"
format_stamp: "Formatted at 2026-08-08 11:42:59 on dist-test-slave-tc2s"
server_key: "98fe09da5e7ff2f8f385f163e2d18ccb"
server_key_iv: "a5305162817785b0d35d07bcfc80426a"
server_key_version: "encryptionkey@0"
I20260808 11:42:59.391870 25962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:42:59.404806 25962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:42:59.410128 25962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:42:59.410362 25962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:42:59.426468 25962 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44797
I20260808 11:42:59.426486 26073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44797 every 8 connection(s)
I20260808 11:42:59.427733 25962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:42:59.431244 26074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:42:59.432337 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 25962
I20260808 11:42:59.432540 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:42:59.433097 24881 external_mini_cluster.cc:1428] Setting key b2d423f07455d8d2d9afdb49c8fba6e1
I20260808 11:42:59.443641 26074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06ac98ac71b14709985f26622cc71589: Bootstrap starting.
I20260808 11:42:59.445691 26077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:59.434925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.137:39961 (local address 127.0.0.1:44797)
0808 11:42:59.435675 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:59.435690 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:59.435733 (+ 43us) server_negotiation.cc:407] Connection header received
0808 11:42:59.437475 (+ 1742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:59.437496 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:59.437868 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:59.438269 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:59.438830 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:59.440067 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:59.441380 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:59.442068 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:59.442252 (+ 184us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:59.443546 (+ 1294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:59.443554 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:59.443567 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:42:59.444111 (+ 544us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:59.444186 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:42:59.444211 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:59.444357 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:59.444642 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 11:42:59.444787 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":308,"threads_started":1}
I20260808 11:42:59.446704 26074 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06ac98ac71b14709985f26622cc71589: Neither blocks nor log segments found. Creating new log.
I20260808 11:42:59.447960 26074 log.cc:824] T 00000000000000000000000000000000 P 06ac98ac71b14709985f26622cc71589: Log is configured to *not* fsync() on all Append() calls
I20260808 11:42:59.449536 24881 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:42:59.450759 26074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06ac98ac71b14709985f26622cc71589: No bootstrap required, opened a new log
I20260808 11:42:59.459352 26074 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06ac98ac71b14709985f26622cc71589 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06ac98ac71b14709985f26622cc71589" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44797 } }
I20260808 11:42:59.459781 26074 raft_consensus.cc:374] T 00000000000000000000000000000000 P 06ac98ac71b14709985f26622cc71589 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:42:59.459896 26074 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06ac98ac71b14709985f26622cc71589 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ac98ac71b14709985f26622cc71589, State: Initialized, Role: FOLLOWER
I20260808 11:42:59.460390 26077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:42:59.453935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.137:52955 (local address 127.0.0.1:44797)
0808 11:42:59.454119 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 11:42:59.454124 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:42:59.454416 (+ 292us) server_negotiation.cc:407] Connection header received
0808 11:42:59.454564 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:42:59.454573 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:42:59.454652 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:42:59.454767 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:42:59.455292 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:59.456428 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:59.457773 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:42:59.458487 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:42:59.458625 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:42:59.459605 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:42:59.459612 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:42:59.459617 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:42:59.459637 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:42:59.459700 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:42:59.459722 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:42:59.459843 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 11:42:59.460023 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 11:42:59.460075 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 11:42:59.460646 26074 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06ac98ac71b14709985f26622cc71589 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06ac98ac71b14709985f26622cc71589" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44797 } }
I20260808 11:42:59.460958 26074 raft_consensus.cc:388] T 00000000000000000000000000000000 P 06ac98ac71b14709985f26622cc71589 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:42:59.461112 26074 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06ac98ac71b14709985f26622cc71589 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:42:59.461298 26074 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06ac98ac71b14709985f26622cc71589 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:42:59.463533 26074 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06ac98ac71b14709985f26622cc71589 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06ac98ac71b14709985f26622cc71589" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44797 } }
I20260808 11:42:59.463964 26074 leader_election.cc:302] T 00000000000000000000000000000000 P 06ac98ac71b14709985f26622cc71589 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 06ac98ac71b14709985f26622cc71589; no voters:
I20260808 11:42:59.464519 26074 leader_election.cc:288] T 00000000000000000000000000000000 P 06ac98ac71b14709985f26622cc71589 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:42:59.464630 26087 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06ac98ac71b14709985f26622cc71589 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:42:59.465802 26087 raft_consensus.cc:686] T 00000000000000000000000000000000 P 06ac98ac71b14709985f26622cc71589 [term 1 LEADER]: Becoming Leader. State: Replica: 06ac98ac71b14709985f26622cc71589, State: Running, Role: LEADER
I20260808 11:42:59.466424 26087 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06ac98ac71b14709985f26622cc71589 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06ac98ac71b14709985f26622cc71589" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44797 } }
I20260808 11:42:59.466671 26074 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06ac98ac71b14709985f26622cc71589 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:42:59.468958 26092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06ac98ac71b14709985f26622cc71589 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06ac98ac71b14709985f26622cc71589. Latest consensus state: current_term: 1 leader_uuid: "06ac98ac71b14709985f26622cc71589" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06ac98ac71b14709985f26622cc71589" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44797 } } }
I20260808 11:42:59.468971 26090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06ac98ac71b14709985f26622cc71589 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "06ac98ac71b14709985f26622cc71589" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06ac98ac71b14709985f26622cc71589" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44797 } } }
I20260808 11:42:59.469375 26092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06ac98ac71b14709985f26622cc71589 [sys.catalog]: This master's current role is: LEADER
I20260808 11:42:59.469421 26090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06ac98ac71b14709985f26622cc71589 [sys.catalog]: This master's current role is: LEADER
I20260808 11:42:59.480852 26096 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:42:59.484830 26096 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:42:59.492421 26096 catalog_manager.cc:1347] Generated new cluster ID: 85de565571724ce9b8c1f3a63e14e7a3
I20260808 11:42:59.492518 26096 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:42:59.508302 26096 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:42:59.509685 26096 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:42:59.516170 26096 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 06ac98ac71b14709985f26622cc71589: Generated new TSK 0
I20260808 11:42:59.516897 26096 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:42:59.524259 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 25962
W20260808 11:42:59.525274 26069 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.137:39961
W20260808 11:42:59.526060 24881 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 25962: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:42:59.526188 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 25962
W20260808 11:42:59.550760 25943 connection.cc:535] client connection to 127.0.0.1:44797 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44797): Connection reset by peer (error 104)
2026-08-08T11:42:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2047 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:42:59 dist-test-slave-tc2s krb5kdc[26121](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:42:59 dist-test-slave-tc2s krb5kdc[26121](info): set up 2 sockets
Aug 08 11:42:59 dist-test-slave-tc2s krb5kdc[26121](info): commencing operation
krb5kdc: starting...
W20260808 11:43:00.747277 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:43:00 dist-test-slave-tc2s krb5kdc[26121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:43:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:43:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 11:43:01.039178 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41235
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41235
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:43:01.369679 26142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:43:01.370002 26142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:43:01.370142 26142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:43:01.377146 26142 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:43:01.377265 26142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:43:01.377359 26142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:43:01.377444 26142 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:43:01.377507 26142 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:43:01.387126 26142 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41235
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:43:01.388386 26142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:43:01.389652 26142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:43:01.401213 26148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:01.401182 26149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:01.401240 26152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:43:01.401960 26142 server_base.cc:1034] running on GCE node
I20260808 11:43:01.402662 26142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:43:01.406165 26142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:43:01.407414 26142 hybrid_clock.cc:648] HybridClock initialized: now 1786189381407347 us; error 44 us; skew 500 ppm
I20260808 11:43:01.407814 26142 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:43:01 dist-test-slave-tc2s krb5kdc[26121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189381, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:43:01.411808 26142 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 11:43:01.413335 26142 webserver.cc:460] Webserver started at http://127.0.0.1:40781/ using document root <none> and password file <none>
I20260808 11:43:01.413978 26142 fs_manager.cc:359] Metadata directory not provided
I20260808 11:43:01.414086 26142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:43:01.414331 26142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:43:01.417177 26142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "a7d6159be128438b87cea58c60f26553"
format_stamp: "Formatted at 2026-08-08 11:43:01 on dist-test-slave-tc2s"
server_key: "1f107aacbab4ffe7add90bbcbb138a79"
server_key_iv: "6406255c9cc32d1efdac1f9de2cf7366"
server_key_version: "encryptionkey@0"
I20260808 11:43:01.417886 26142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "a7d6159be128438b87cea58c60f26553"
format_stamp: "Formatted at 2026-08-08 11:43:01 on dist-test-slave-tc2s"
server_key: "1f107aacbab4ffe7add90bbcbb138a79"
server_key_iv: "6406255c9cc32d1efdac1f9de2cf7366"
server_key_version: "encryptionkey@0"
I20260808 11:43:01.422866 26142 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 11:43:01.426050 26171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:01.427239 26142 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 11:43:01.427387 26142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "a7d6159be128438b87cea58c60f26553"
format_stamp: "Formatted at 2026-08-08 11:43:01 on dist-test-slave-tc2s"
server_key: "1f107aacbab4ffe7add90bbcbb138a79"
server_key_iv: "6406255c9cc32d1efdac1f9de2cf7366"
server_key_version: "encryptionkey@0"
I20260808 11:43:01.427561 26142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:43:01.432930 26142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:43:01.436915 26142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:43:01.437175 26142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:43:01.454196 26142 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41235
I20260808 11:43:01.454267 26257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41235 every 8 connection(s)
I20260808 11:43:01.455425 26142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:43:01.455763 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 26142
I20260808 11:43:01.455966 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:43:01.456532 24881 external_mini_cluster.cc:1428] Setting key 353a5086909ed5cd87f321969139a053
I20260808 11:43:01.459473 26260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:01.463847 26263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:01.458201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.137:53319 (local address 127.0.0.1:41235)
0808 11:43:01.458923 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:01.458935 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:01.458976 (+ 41us) server_negotiation.cc:407] Connection header received
0808 11:43:01.460773 (+ 1797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:01.460795 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:01.461098 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:01.461388 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:01.462400 (+ 1012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:01.462406 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:01.462418 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:43:01.462463 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:01.462541 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:43:01.462560 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:01.462721 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:01.462816 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 11:43:01.462972 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":327,"threads_started":1}
I20260808 11:43:01.467324 24881 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:43:01.470724 26260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7d6159be128438b87cea58c60f26553: Bootstrap starting.
I20260808 11:43:01.473658 26263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:01.471499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.137:57373 (local address 127.0.0.1:41235)
0808 11:43:01.471640 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:01.471646 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:01.472026 (+ 380us) server_negotiation.cc:407] Connection header received
0808 11:43:01.472181 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:01.472185 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:01.472229 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:01.472305 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:01.473098 (+ 793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:01.473104 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:01.473109 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:43:01.473126 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:01.473160 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:43:01.473173 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:01.473265 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:01.473361 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 11:43:01.473414 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 11:43:01.473841 26260 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7d6159be128438b87cea58c60f26553: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:01.474931 26260 log.cc:824] T 00000000000000000000000000000000 P a7d6159be128438b87cea58c60f26553: Log is configured to *not* fsync() on all Append() calls
I20260808 11:43:01.477342 26260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7d6159be128438b87cea58c60f26553: No bootstrap required, opened a new log
I20260808 11:43:01.484387 26260 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7d6159be128438b87cea58c60f26553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7d6159be128438b87cea58c60f26553" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41235 } }
I20260808 11:43:01.484704 26260 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7d6159be128438b87cea58c60f26553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:01.484776 26260 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7d6159be128438b87cea58c60f26553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7d6159be128438b87cea58c60f26553, State: Initialized, Role: FOLLOWER
I20260808 11:43:01.485250 26260 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7d6159be128438b87cea58c60f26553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7d6159be128438b87cea58c60f26553" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41235 } }
I20260808 11:43:01.485455 26260 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7d6159be128438b87cea58c60f26553 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:43:01.485615 26260 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7d6159be128438b87cea58c60f26553 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:43:01.485814 26260 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7d6159be128438b87cea58c60f26553 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:01.487912 26260 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7d6159be128438b87cea58c60f26553 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7d6159be128438b87cea58c60f26553" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41235 } }
I20260808 11:43:01.488348 26260 leader_election.cc:302] T 00000000000000000000000000000000 P a7d6159be128438b87cea58c60f26553 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7d6159be128438b87cea58c60f26553; no voters:
I20260808 11:43:01.488847 26260 leader_election.cc:288] T 00000000000000000000000000000000 P a7d6159be128438b87cea58c60f26553 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:43:01.488967 26272 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7d6159be128438b87cea58c60f26553 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:43:01.489840 26272 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7d6159be128438b87cea58c60f26553 [term 1 LEADER]: Becoming Leader. State: Replica: a7d6159be128438b87cea58c60f26553, State: Running, Role: LEADER
I20260808 11:43:01.490525 26272 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7d6159be128438b87cea58c60f26553 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7d6159be128438b87cea58c60f26553" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41235 } }
I20260808 11:43:01.490784 26260 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7d6159be128438b87cea58c60f26553 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:43:01.493431 26278 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7d6159be128438b87cea58c60f26553 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7d6159be128438b87cea58c60f26553. Latest consensus state: current_term: 1 leader_uuid: "a7d6159be128438b87cea58c60f26553" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7d6159be128438b87cea58c60f26553" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41235 } } }
I20260808 11:43:01.493400 26273 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7d6159be128438b87cea58c60f26553 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7d6159be128438b87cea58c60f26553" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7d6159be128438b87cea58c60f26553" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41235 } } }
I20260808 11:43:01.493871 26278 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7d6159be128438b87cea58c60f26553 [sys.catalog]: This master's current role is: LEADER
I20260808 11:43:01.493871 26273 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7d6159be128438b87cea58c60f26553 [sys.catalog]: This master's current role is: LEADER
I20260808 11:43:01.495338 26286 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:43:01.500408 26286 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:43:01.508965 26286 catalog_manager.cc:1347] Generated new cluster ID: 7474a6454e1340f988dd7c905954c127
I20260808 11:43:01.509075 26286 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:43:01.566144 26286 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:43:01.566561 26286 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:43:01.575057 26286 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7d6159be128438b87cea58c60f26553: Generated new TSK 0
I20260808 11:43:01.575793 26286 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:43:01.580998 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 26142
W20260808 11:43:01.582095 26250 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.137:53319
W20260808 11:43:01.582712 24881 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 26142: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:43:01.582841 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 26142
2026-08-08T11:43:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2056 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:43:01 dist-test-slave-tc2s krb5kdc[26302](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:43:01 dist-test-slave-tc2s krb5kdc[26302](info): set up 2 sockets
Aug 08 11:43:01 dist-test-slave-tc2s krb5kdc[26302](info): commencing operation
krb5kdc: starting...
W20260808 11:43:02.801767 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.002s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:43:02 dist-test-slave-tc2s krb5kdc[26302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:43:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:43:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 11:43:03.099097 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37873
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:43:03.440513 26327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:43:03.440778 26327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:43:03.440902 26327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:43:03.447887 26327 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:43:03.447986 26327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:43:03.448055 26327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:43:03.448130 26327 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:43:03.448187 26327 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:43:03.457594 26327 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37873
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:43:03.458799 26327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:43:03.460011 26327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:43:03.471410 26334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:03.471501 26332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:03.471468 26337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:43:03.472318 26327 server_base.cc:1034] running on GCE node
I20260808 11:43:03.473024 26327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:43:03.474330 26327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:43:03.475521 26327 hybrid_clock.cc:648] HybridClock initialized: now 1786189383475479 us; error 30 us; skew 500 ppm
I20260808 11:43:03.475821 26327 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:43:03 dist-test-slave-tc2s krb5kdc[26302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189383, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:43:03.490612 26327 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 11:43:03.492043 26327 webserver.cc:460] Webserver started at http://127.0.0.1:37453/ using document root <none> and password file <none>
I20260808 11:43:03.492694 26327 fs_manager.cc:359] Metadata directory not provided
I20260808 11:43:03.492810 26327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:43:03.493105 26327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:43:03.496064 26327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "5ac1e7cb52284a2bb2cdde93e773b93f"
format_stamp: "Formatted at 2026-08-08 11:43:03 on dist-test-slave-tc2s"
server_key: "3f6358e985fb047b72dc377fd79eb6a4"
server_key_iv: "f1518db169c41cdfa28315bf75b24503"
server_key_version: "encryptionkey@0"
I20260808 11:43:03.496826 26327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "5ac1e7cb52284a2bb2cdde93e773b93f"
format_stamp: "Formatted at 2026-08-08 11:43:03 on dist-test-slave-tc2s"
server_key: "3f6358e985fb047b72dc377fd79eb6a4"
server_key_iv: "f1518db169c41cdfa28315bf75b24503"
server_key_version: "encryptionkey@0"
I20260808 11:43:03.501866 26327 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260808 11:43:03.505127 26351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:03.506371 26327 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:43:03.506523 26327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "5ac1e7cb52284a2bb2cdde93e773b93f"
format_stamp: "Formatted at 2026-08-08 11:43:03 on dist-test-slave-tc2s"
server_key: "3f6358e985fb047b72dc377fd79eb6a4"
server_key_iv: "f1518db169c41cdfa28315bf75b24503"
server_key_version: "encryptionkey@0"
I20260808 11:43:03.506700 26327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:43:03.523810 26327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:43:03.527578 26327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:43:03.527833 26327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:43:03.544876 26327 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37873
I20260808 11:43:03.544903 26434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37873 every 8 connection(s)
I20260808 11:43:03.546183 26327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:43:03.549755 26435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:03.551862 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 26327
I20260808 11:43:03.552037 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:43:03.552600 24881 external_mini_cluster.cc:1428] Setting key 154972c3afd12e5158f61d55fdb49c8e
I20260808 11:43:03.564015 26435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ac1e7cb52284a2bb2cdde93e773b93f: Bootstrap starting.
Aug 08 11:43:03 dist-test-slave-tc2s krb5kdc[26302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 11:43:03.567099 26435 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ac1e7cb52284a2bb2cdde93e773b93f: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:03.568202 26435 log.cc:824] T 00000000000000000000000000000000 P 5ac1e7cb52284a2bb2cdde93e773b93f: Log is configured to *not* fsync() on all Append() calls
I20260808 11:43:03.571736 26435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ac1e7cb52284a2bb2cdde93e773b93f: No bootstrap required, opened a new log
I20260808 11:43:03.576413 26439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:03.554429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.137:35045 (local address 127.0.0.1:37873)
0808 11:43:03.555203 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:03.555218 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:03.555276 (+ 58us) server_negotiation.cc:407] Connection header received
0808 11:43:03.556989 (+ 1713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:03.557022 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:03.557549 (+ 527us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:03.557979 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:03.559328 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:03.561397 (+ 2069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:03.562736 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:03.563726 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:03.563902 (+ 176us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:03.568038 (+ 4136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:03.568049 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:03.568067 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:03.568110 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:03.571779 (+ 3669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:03.572647 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:03.572655 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:03.572663 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:03.572880 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:03.573286 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:03.573292 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:03.573295 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:03.574028 (+ 733us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:03.574361 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:03.574800 (+ 439us) server_negotiation.cc:299] Negotiation successful
0808 11:43:03.575141 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":308,"threads_started":1}
I20260808 11:43:03.580212 24881 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:43:03.580797 26435 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ac1e7cb52284a2bb2cdde93e773b93f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ac1e7cb52284a2bb2cdde93e773b93f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37873 } }
I20260808 11:43:03.581203 26435 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ac1e7cb52284a2bb2cdde93e773b93f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:03.581293 26435 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ac1e7cb52284a2bb2cdde93e773b93f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ac1e7cb52284a2bb2cdde93e773b93f, State: Initialized, Role: FOLLOWER
I20260808 11:43:03.581872 26435 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ac1e7cb52284a2bb2cdde93e773b93f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ac1e7cb52284a2bb2cdde93e773b93f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37873 } }
I20260808 11:43:03.582170 26435 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ac1e7cb52284a2bb2cdde93e773b93f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:43:03.582298 26435 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ac1e7cb52284a2bb2cdde93e773b93f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:43:03.582463 26435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ac1e7cb52284a2bb2cdde93e773b93f [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:03.585354 26435 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ac1e7cb52284a2bb2cdde93e773b93f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ac1e7cb52284a2bb2cdde93e773b93f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37873 } }
I20260808 11:43:03.585831 26435 leader_election.cc:302] T 00000000000000000000000000000000 P 5ac1e7cb52284a2bb2cdde93e773b93f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ac1e7cb52284a2bb2cdde93e773b93f; no voters:
I20260808 11:43:03.586508 26435 leader_election.cc:288] T 00000000000000000000000000000000 P 5ac1e7cb52284a2bb2cdde93e773b93f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:43:03.586637 26451 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ac1e7cb52284a2bb2cdde93e773b93f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:43:03.587693 26451 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ac1e7cb52284a2bb2cdde93e773b93f [term 1 LEADER]: Becoming Leader. State: Replica: 5ac1e7cb52284a2bb2cdde93e773b93f, State: Running, Role: LEADER
I20260808 11:43:03.588351 26451 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ac1e7cb52284a2bb2cdde93e773b93f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ac1e7cb52284a2bb2cdde93e773b93f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37873 } }
I20260808 11:43:03.588682 26435 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ac1e7cb52284a2bb2cdde93e773b93f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:43:03.600592 26454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ac1e7cb52284a2bb2cdde93e773b93f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ac1e7cb52284a2bb2cdde93e773b93f. Latest consensus state: current_term: 1 leader_uuid: "5ac1e7cb52284a2bb2cdde93e773b93f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ac1e7cb52284a2bb2cdde93e773b93f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37873 } } }
I20260808 11:43:03.601001 26454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ac1e7cb52284a2bb2cdde93e773b93f [sys.catalog]: This master's current role is: LEADER
I20260808 11:43:03.601552 26453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ac1e7cb52284a2bb2cdde93e773b93f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ac1e7cb52284a2bb2cdde93e773b93f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ac1e7cb52284a2bb2cdde93e773b93f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37873 } } }
I20260808 11:43:03.601852 26453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ac1e7cb52284a2bb2cdde93e773b93f [sys.catalog]: This master's current role is: LEADER
I20260808 11:43:03.602353 26439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:03.585261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.137:45755 (local address 127.0.0.1:37873)
0808 11:43:03.585489 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:03.585494 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:03.585944 (+ 450us) server_negotiation.cc:407] Connection header received
0808 11:43:03.586119 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:03.586124 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:03.586200 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:03.586305 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:03.588309 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:03.589437 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:03.591345 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:03.592055 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:03.592189 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:03.595446 (+ 3257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:03.595453 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:03.595458 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:03.595508 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:03.599412 (+ 3904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:03.600163 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:03.600168 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:03.600170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:03.600299 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:03.600750 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:03.600757 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:03.600759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:03.601133 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:03.601393 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:03.601815 (+ 422us) server_negotiation.cc:299] Negotiation successful
0808 11:43:03.602008 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260808 11:43:03.605154 26464 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:43:03.610165 26464 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:43:03.618889 26464 catalog_manager.cc:1347] Generated new cluster ID: 7865d4e0dfb3421582a05ee07b1202eb
I20260808 11:43:03.619004 26464 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:43:03.635533 26464 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:43:03.636713 26464 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:43:03.645051 26464 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ac1e7cb52284a2bb2cdde93e773b93f: Generated new TSK 0
I20260808 11:43:03.645730 26464 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:43:03.651955 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 26327
I20260808 11:43:03.653278 26425 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:43:03.741789 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 26327
W20260808 11:43:03.765507 26299 connection.cc:535] client connection to 127.0.0.1:37873 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37873): Connection reset by peer (error 104)
2026-08-08T11:43:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2157 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:43:03 dist-test-slave-tc2s krb5kdc[26480](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:43:03 dist-test-slave-tc2s krb5kdc[26480](info): set up 2 sockets
Aug 08 11:43:03 dist-test-slave-tc2s krb5kdc[26480](info): commencing operation
krb5kdc: starting...
W20260808 11:43:04.967324 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:43:05 dist-test-slave-tc2s krb5kdc[26480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:43:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:43:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 11:43:05.263330 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35145
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35145
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:43:05.591217 26498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:43:05.591506 26498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:43:05.591648 26498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:43:05.598896 26498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:43:05.599004 26498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:43:05.599108 26498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:43:05.599187 26498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:43:05.599249 26498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:43:05.608573 26498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35145
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:43:05.609766 26498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:43:05.610989 26498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:43:05.622012 26506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:05.622044 26509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:05.622031 26507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:43:05.623246 26498 server_base.cc:1034] running on GCE node
I20260808 11:43:05.623940 26498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:43:05.625190 26498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:43:05.626385 26498 hybrid_clock.cc:648] HybridClock initialized: now 1786189385626349 us; error 25 us; skew 500 ppm
I20260808 11:43:05.626698 26498 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:43:05 dist-test-slave-tc2s krb5kdc[26480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189385, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:43:05.629588 26498 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 11:43:05.630975 26498 webserver.cc:460] Webserver started at http://127.0.0.1:39121/ using document root <none> and password file <none>
I20260808 11:43:05.631588 26498 fs_manager.cc:359] Metadata directory not provided
I20260808 11:43:05.631701 26498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:43:05.631983 26498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:43:05.634835 26498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "13fc522a03894e8ca5d75f49571eeb2a"
format_stamp: "Formatted at 2026-08-08 11:43:05 on dist-test-slave-tc2s"
server_key: "99f568e945be3ae90a6011cd76935b98"
server_key_iv: "586651315ff609cae351fe8a4a4ad477"
server_key_version: "encryptionkey@0"
I20260808 11:43:05.635541 26498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "13fc522a03894e8ca5d75f49571eeb2a"
format_stamp: "Formatted at 2026-08-08 11:43:05 on dist-test-slave-tc2s"
server_key: "99f568e945be3ae90a6011cd76935b98"
server_key_iv: "586651315ff609cae351fe8a4a4ad477"
server_key_version: "encryptionkey@0"
I20260808 11:43:05.640205 26498 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 11:43:05.643373 26518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:05.644531 26498 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 11:43:05.644685 26498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "13fc522a03894e8ca5d75f49571eeb2a"
format_stamp: "Formatted at 2026-08-08 11:43:05 on dist-test-slave-tc2s"
server_key: "99f568e945be3ae90a6011cd76935b98"
server_key_iv: "586651315ff609cae351fe8a4a4ad477"
server_key_version: "encryptionkey@0"
I20260808 11:43:05.644861 26498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:43:05.671244 26498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:43:05.674881 26498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:43:05.675138 26498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:43:05.691943 26498 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35145
I20260808 11:43:05.692003 26591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35145 every 8 connection(s)
I20260808 11:43:05.693204 26498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:43:05.696672 26592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:05.698830 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 26498
I20260808 11:43:05.699011 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:43:05.699554 24881 external_mini_cluster.cc:1428] Setting key b3df42c36f9410c3204a3be75cb971b2
I20260808 11:43:05.709319 26592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13fc522a03894e8ca5d75f49571eeb2a: Bootstrap starting.
I20260808 11:43:05.712877 26592 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13fc522a03894e8ca5d75f49571eeb2a: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:05.714192 26592 log.cc:824] T 00000000000000000000000000000000 P 13fc522a03894e8ca5d75f49571eeb2a: Log is configured to *not* fsync() on all Append() calls
I20260808 11:43:05.714535 26595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:05.701397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.137:35027 (local address 127.0.0.1:35145)
0808 11:43:05.702301 (+ 904us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:05.702316 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:05.702665 (+ 349us) server_negotiation.cc:407] Connection header received
0808 11:43:05.704997 (+ 2332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:05.705025 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:05.705512 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:05.705954 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:05.706458 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:05.708410 (+ 1952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:05.709706 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:05.710704 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:05.710906 (+ 202us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:05.711949 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:05.711961 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:05.711989 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:43:05.712637 (+ 648us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:05.712732 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:43:05.712769 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:05.712960 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:05.713128 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 11:43:05.713296 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":397,"threads_started":1}
I20260808 11:43:05.716975 26592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13fc522a03894e8ca5d75f49571eeb2a: No bootstrap required, opened a new log
I20260808 11:43:05.718218 24881 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:43:05.726467 26592 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13fc522a03894e8ca5d75f49571eeb2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13fc522a03894e8ca5d75f49571eeb2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35145 } }
I20260808 11:43:05.726807 26592 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13fc522a03894e8ca5d75f49571eeb2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:05.726912 26592 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13fc522a03894e8ca5d75f49571eeb2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13fc522a03894e8ca5d75f49571eeb2a, State: Initialized, Role: FOLLOWER
I20260808 11:43:05.727463 26592 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13fc522a03894e8ca5d75f49571eeb2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13fc522a03894e8ca5d75f49571eeb2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35145 } }
I20260808 11:43:05.727712 26592 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13fc522a03894e8ca5d75f49571eeb2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:43:05.727837 26592 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13fc522a03894e8ca5d75f49571eeb2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:43:05.727995 26592 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13fc522a03894e8ca5d75f49571eeb2a [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:05.729346 26595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:05.723155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.137:44369 (local address 127.0.0.1:35145)
0808 11:43:05.723317 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:05.723322 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:05.723746 (+ 424us) server_negotiation.cc:407] Connection header received
0808 11:43:05.723874 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:05.723879 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:05.723945 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:05.724047 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:05.724435 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:05.725590 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:05.726889 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:05.727375 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:05.727485 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:05.728628 (+ 1143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:05.728637 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:05.728640 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:43:05.728661 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:05.728714 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:43:05.728734 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:05.728837 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:05.729043 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 11:43:05.729099 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 11:43:05.730445 26592 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13fc522a03894e8ca5d75f49571eeb2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13fc522a03894e8ca5d75f49571eeb2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35145 } }
I20260808 11:43:05.730945 26592 leader_election.cc:302] T 00000000000000000000000000000000 P 13fc522a03894e8ca5d75f49571eeb2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13fc522a03894e8ca5d75f49571eeb2a; no voters:
I20260808 11:43:05.731434 26592 leader_election.cc:288] T 00000000000000000000000000000000 P 13fc522a03894e8ca5d75f49571eeb2a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:43:05.731549 26614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13fc522a03894e8ca5d75f49571eeb2a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:43:05.732183 26614 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13fc522a03894e8ca5d75f49571eeb2a [term 1 LEADER]: Becoming Leader. State: Replica: 13fc522a03894e8ca5d75f49571eeb2a, State: Running, Role: LEADER
I20260808 11:43:05.732659 26614 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13fc522a03894e8ca5d75f49571eeb2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13fc522a03894e8ca5d75f49571eeb2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35145 } }
I20260808 11:43:05.733374 26592 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13fc522a03894e8ca5d75f49571eeb2a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:43:05.735154 26617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13fc522a03894e8ca5d75f49571eeb2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13fc522a03894e8ca5d75f49571eeb2a. Latest consensus state: current_term: 1 leader_uuid: "13fc522a03894e8ca5d75f49571eeb2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13fc522a03894e8ca5d75f49571eeb2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35145 } } }
I20260808 11:43:05.735190 26615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13fc522a03894e8ca5d75f49571eeb2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13fc522a03894e8ca5d75f49571eeb2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13fc522a03894e8ca5d75f49571eeb2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35145 } } }
I20260808 11:43:05.735616 26615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13fc522a03894e8ca5d75f49571eeb2a [sys.catalog]: This master's current role is: LEADER
I20260808 11:43:05.737596 26617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13fc522a03894e8ca5d75f49571eeb2a [sys.catalog]: This master's current role is: LEADER
I20260808 11:43:05.743686 26623 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:43:05.747609 26623 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:43:05.755738 26623 catalog_manager.cc:1347] Generated new cluster ID: 544834d9a79b416d801d7ff4cb7656c0
I20260808 11:43:05.755842 26623 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:43:05.767753 26623 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:43:05.769002 26623 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:43:05.774341 26623 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13fc522a03894e8ca5d75f49571eeb2a: Generated new TSK 0
I20260808 11:43:05.775038 26623 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:43:05.784435 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 26498
W20260808 11:43:05.785542 26584 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.137:35027
W20260808 11:43:05.786307 24881 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 26498: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:43:05.786434 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 26498
W20260808 11:43:05.813344 26476 connection.cc:535] client connection to 127.0.0.1:35145 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35145): Connection reset by peer (error 104)
2026-08-08T11:43:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2047 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:43:05 dist-test-slave-tc2s krb5kdc[26650](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:43:05 dist-test-slave-tc2s krb5kdc[26650](info): set up 2 sockets
Aug 08 11:43:05 dist-test-slave-tc2s krb5kdc[26650](info): commencing operation
krb5kdc: starting...
W20260808 11:43:07.016508 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.001s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:43:07 dist-test-slave-tc2s krb5kdc[26650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:43:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:43:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 11:43:07.326527 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34757
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34757
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:43:07.656455 26672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:43:07.656734 26672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:43:07.656831 26672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:43:07.663779 26672 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:43:07.663879 26672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:43:07.663931 26672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:43:07.663997 26672 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:43:07.664041 26672 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:43:07.673518 26672 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34757
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189042529715-24881-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:43:07.674784 26672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:43:07.676026 26672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:43:07.687170 26679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:07.687222 26682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:07.687247 26680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:43:07.688244 26672 server_base.cc:1034] running on GCE node
I20260808 11:43:07.688910 26672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:43:07.690205 26672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:43:07.691407 26672 hybrid_clock.cc:648] HybridClock initialized: now 1786189387691361 us; error 32 us; skew 500 ppm
I20260808 11:43:07.691742 26672 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:43:07 dist-test-slave-tc2s krb5kdc[26650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189387, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:43:07.694726 26672 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 11:43:07.696096 26672 webserver.cc:460] Webserver started at http://127.0.0.1:45717/ using document root <none> and password file <none>
I20260808 11:43:07.696713 26672 fs_manager.cc:359] Metadata directory not provided
I20260808 11:43:07.696825 26672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:43:07.697111 26672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:43:07.699968 26672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "7029b02937744758b9d0c19a62ff8fd7"
format_stamp: "Formatted at 2026-08-08 11:43:07 on dist-test-slave-tc2s"
server_key: "5a0f1f2c94e9c45bb1d62b3b918497ba"
server_key_iv: "3add5623757e12ec86679054949ec94b"
server_key_version: "encryptionkey@0"
I20260808 11:43:07.700699 26672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "7029b02937744758b9d0c19a62ff8fd7"
format_stamp: "Formatted at 2026-08-08 11:43:07 on dist-test-slave-tc2s"
server_key: "5a0f1f2c94e9c45bb1d62b3b918497ba"
server_key_iv: "3add5623757e12ec86679054949ec94b"
server_key_version: "encryptionkey@0"
I20260808 11:43:07.705513 26672 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 11:43:07.708727 26703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:07.709967 26672 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260808 11:43:07.710124 26672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "7029b02937744758b9d0c19a62ff8fd7"
format_stamp: "Formatted at 2026-08-08 11:43:07 on dist-test-slave-tc2s"
server_key: "5a0f1f2c94e9c45bb1d62b3b918497ba"
server_key_iv: "3add5623757e12ec86679054949ec94b"
server_key_version: "encryptionkey@0"
I20260808 11:43:07.710294 26672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:43:07.715655 26672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:43:07.719249 26672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:43:07.719511 26672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:43:07.736290 26672 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34757
I20260808 11:43:07.736347 26777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34757 every 8 connection(s)
I20260808 11:43:07.737555 26672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:43:07.741074 26779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:07.744346 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 26672
I20260808 11:43:07.744509 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:43:07.744920 24881 external_mini_cluster.cc:1428] Setting key 70253506bec3ee719bfc0111bbaebd90
I20260808 11:43:07.753232 26782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:07.746659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.137:38213 (local address 127.0.0.1:34757)
0808 11:43:07.747430 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:07.747444 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:07.747511 (+ 67us) server_negotiation.cc:407] Connection header received
0808 11:43:07.749478 (+ 1967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:07.749516 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:07.749979 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:07.750406 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:07.751377 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:07.751385 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:07.751406 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:43:07.751432 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:07.751506 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:43:07.751537 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:07.751761 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:07.751899 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 11:43:07.752090 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":300,"threads_started":1}
I20260808 11:43:07.754027 26779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7029b02937744758b9d0c19a62ff8fd7: Bootstrap starting.
I20260808 11:43:07.756682 24881 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:43:07.757136 26779 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7029b02937744758b9d0c19a62ff8fd7: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:07.758337 26779 log.cc:824] T 00000000000000000000000000000000 P 7029b02937744758b9d0c19a62ff8fd7: Log is configured to *not* fsync() on all Append() calls
I20260808 11:43:07.761475 26779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7029b02937744758b9d0c19a62ff8fd7: No bootstrap required, opened a new log
I20260808 11:43:07.763976 26782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:07.761567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.137:60835 (local address 127.0.0.1:34757)
0808 11:43:07.761760 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:07.761767 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:07.762148 (+ 381us) server_negotiation.cc:407] Connection header received
0808 11:43:07.762315 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:07.762318 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:07.762365 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:07.762440 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:07.763358 (+ 918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:07.763361 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:07.763365 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:43:07.763382 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:07.763420 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:43:07.763437 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:07.763531 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:07.763663 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 11:43:07.763709 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 11:43:07.769161 26779 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7029b02937744758b9d0c19a62ff8fd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7029b02937744758b9d0c19a62ff8fd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34757 } }
I20260808 11:43:07.769505 26779 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7029b02937744758b9d0c19a62ff8fd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:07.769601 26779 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7029b02937744758b9d0c19a62ff8fd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7029b02937744758b9d0c19a62ff8fd7, State: Initialized, Role: FOLLOWER
I20260808 11:43:07.770226 26779 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7029b02937744758b9d0c19a62ff8fd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7029b02937744758b9d0c19a62ff8fd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34757 } }
I20260808 11:43:07.770495 26779 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7029b02937744758b9d0c19a62ff8fd7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:43:07.770630 26779 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7029b02937744758b9d0c19a62ff8fd7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:43:07.770783 26779 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7029b02937744758b9d0c19a62ff8fd7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:07.773020 26779 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7029b02937744758b9d0c19a62ff8fd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7029b02937744758b9d0c19a62ff8fd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34757 } }
I20260808 11:43:07.773476 26779 leader_election.cc:302] T 00000000000000000000000000000000 P 7029b02937744758b9d0c19a62ff8fd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7029b02937744758b9d0c19a62ff8fd7; no voters:
I20260808 11:43:07.774137 26779 leader_election.cc:288] T 00000000000000000000000000000000 P 7029b02937744758b9d0c19a62ff8fd7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:43:07.774257 26794 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7029b02937744758b9d0c19a62ff8fd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:43:07.774932 26794 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7029b02937744758b9d0c19a62ff8fd7 [term 1 LEADER]: Becoming Leader. State: Replica: 7029b02937744758b9d0c19a62ff8fd7, State: Running, Role: LEADER
I20260808 11:43:07.775480 26794 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7029b02937744758b9d0c19a62ff8fd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7029b02937744758b9d0c19a62ff8fd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34757 } }
I20260808 11:43:07.776191 26779 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7029b02937744758b9d0c19a62ff8fd7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:43:07.777987 26796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7029b02937744758b9d0c19a62ff8fd7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7029b02937744758b9d0c19a62ff8fd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7029b02937744758b9d0c19a62ff8fd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34757 } } }
I20260808 11:43:07.777949 26798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7029b02937744758b9d0c19a62ff8fd7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7029b02937744758b9d0c19a62ff8fd7. Latest consensus state: current_term: 1 leader_uuid: "7029b02937744758b9d0c19a62ff8fd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7029b02937744758b9d0c19a62ff8fd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34757 } } }
I20260808 11:43:07.778378 26796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7029b02937744758b9d0c19a62ff8fd7 [sys.catalog]: This master's current role is: LEADER
I20260808 11:43:07.778427 26798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7029b02937744758b9d0c19a62ff8fd7 [sys.catalog]: This master's current role is: LEADER
I20260808 11:43:07.781224 26803 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:43:07.786983 26803 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:43:07.795064 26803 catalog_manager.cc:1347] Generated new cluster ID: c85c4db2520046128dc8821985787c21
I20260808 11:43:07.795168 26803 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:43:07.807569 26803 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:43:07.807966 26803 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:43:07.812881 26803 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7029b02937744758b9d0c19a62ff8fd7: Generated new TSK 0
I20260808 11:43:07.813655 26803 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:43:07.822876 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 26672
W20260808 11:43:07.824244 26771 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.137:38213
W20260808 11:43:07.824873 24881 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 26672: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:43:07.825024 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 26672
2026-08-08T11:43:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2036 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:43:07 dist-test-slave-tc2s krb5kdc[26830](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:43:07 dist-test-slave-tc2s krb5kdc[26830](info): set up 2 sockets
Aug 08 11:43:07 dist-test-slave-tc2s krb5kdc[26830](info): commencing operation
krb5kdc: starting...
W20260808 11:43:09.053272 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.001s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:43:09 dist-test-slave-tc2s krb5kdc[26830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189389, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:43:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:43:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
I20260808 11:43:09.354912 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:32805
--webserver_interface=127.24.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:32935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:32805
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:43:09.693243 26849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:43:09.693600 26849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:43:09.693722 26849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:43:09.700721 26849 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:43:09.700819 26849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:43:09.700870 26849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:43:09.700928 26849 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:43:09.700969 26849 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:43:09.710399 26849 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:32935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:32805
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:32805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:43:09.711553 26849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:43:09.712785 26849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:43:09.724292 26858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:09.724373 26855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:09.724314 26861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:43:09.725481 26849 server_base.cc:1034] running on GCE node
I20260808 11:43:09.726281 26849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:43:09.727572 26849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:43:09.728778 26849 hybrid_clock.cc:648] HybridClock initialized: now 1786189389728722 us; error 43 us; skew 500 ppm
I20260808 11:43:09.729110 26849 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:43:09 dist-test-slave-tc2s krb5kdc[26830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189389, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:43:09.736632 26849 init.cc:377] Logged in from keytab as kudu/127.24.76.126@KRBTEST.COM (short username kudu)
I20260808 11:43:09.738092 26849 webserver.cc:460] Webserver started at http://127.24.76.126:40677/ using document root <none> and password file <none>
I20260808 11:43:09.738756 26849 fs_manager.cc:359] Metadata directory not provided
I20260808 11:43:09.738874 26849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:43:09.739164 26849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:43:09.742106 26849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "ce79b12353984ed19f0e2c394abb4724"
format_stamp: "Formatted at 2026-08-08 11:43:09 on dist-test-slave-tc2s"
server_key: "4bd408361911ed8a86d62189fd6bd1be"
server_key_iv: "66dd5017594069b75beb5955c6834c24"
server_key_version: "encryptionkey@0"
I20260808 11:43:09.742867 26849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "ce79b12353984ed19f0e2c394abb4724"
format_stamp: "Formatted at 2026-08-08 11:43:09 on dist-test-slave-tc2s"
server_key: "4bd408361911ed8a86d62189fd6bd1be"
server_key_iv: "66dd5017594069b75beb5955c6834c24"
server_key_version: "encryptionkey@0"
I20260808 11:43:09.747790 26849 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 11:43:09.751119 26874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:09.752431 26849 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:43:09.752616 26849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "ce79b12353984ed19f0e2c394abb4724"
format_stamp: "Formatted at 2026-08-08 11:43:09 on dist-test-slave-tc2s"
server_key: "4bd408361911ed8a86d62189fd6bd1be"
server_key_iv: "66dd5017594069b75beb5955c6834c24"
server_key_version: "encryptionkey@0"
I20260808 11:43:09.752800 26849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:43:09.788707 26849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:43:09.793030 26849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:43:09.793298 26849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:43:09.810316 26849 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:32805
I20260808 11:43:09.810340 26960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:32805 every 8 connection(s)
I20260808 11:43:09.811733 26849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:43:09.815575 26961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:09.815675 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 26849
I20260808 11:43:09.815836 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:43:09.816285 24881 external_mini_cluster.cc:1428] Setting key 61fe221c333bc7a0acfc0ba3d741fb94
I20260808 11:43:09.829883 26961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce79b12353984ed19f0e2c394abb4724: Bootstrap starting.
Aug 08 11:43:09 dist-test-slave-tc2s krb5kdc[26830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189389, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:43:09.833685 26961 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce79b12353984ed19f0e2c394abb4724: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:09.835105 26961 log.cc:824] T 00000000000000000000000000000000 P ce79b12353984ed19f0e2c394abb4724: Log is configured to *not* fsync() on all Append() calls
I20260808 11:43:09.838588 26961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce79b12353984ed19f0e2c394abb4724: No bootstrap required, opened a new log
I20260808 11:43:09.840354 26966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:09.818047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38455 (local address 127.24.76.126:32805)
0808 11:43:09.818878 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:09.818897 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:09.818951 (+ 54us) server_negotiation.cc:407] Connection header received
0808 11:43:09.820733 (+ 1782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:09.820761 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:09.821288 (+ 527us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:09.821736 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:09.823108 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:09.825025 (+ 1917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:09.826307 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:09.827392 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:09.827600 (+ 208us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:09.831619 (+ 4019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:09.831631 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:09.831651 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:09.831706 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:09.835584 (+ 3878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:09.836648 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:09.836657 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:09.836665 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:09.836879 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:09.837279 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:09.837284 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:09.837287 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:09.837999 (+ 712us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:09.838312 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:09.838685 (+ 373us) server_negotiation.cc:299] Negotiation successful
0808 11:43:09.839045 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":359,"threads_started":1}
I20260808 11:43:09.844506 24881 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:43:09.847587 26961 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce79b12353984ed19f0e2c394abb4724 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce79b12353984ed19f0e2c394abb4724" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 32805 } }
I20260808 11:43:09.847930 26961 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce79b12353984ed19f0e2c394abb4724 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:09.848029 26961 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce79b12353984ed19f0e2c394abb4724 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce79b12353984ed19f0e2c394abb4724, State: Initialized, Role: FOLLOWER
I20260808 11:43:09.848608 26961 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce79b12353984ed19f0e2c394abb4724 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce79b12353984ed19f0e2c394abb4724" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 32805 } }
I20260808 11:43:09.848904 26961 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce79b12353984ed19f0e2c394abb4724 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:43:09.849038 26961 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce79b12353984ed19f0e2c394abb4724 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:43:09.849203 26961 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce79b12353984ed19f0e2c394abb4724 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:09.851670 26961 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce79b12353984ed19f0e2c394abb4724 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce79b12353984ed19f0e2c394abb4724" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 32805 } }
I20260808 11:43:09.852278 26961 leader_election.cc:302] T 00000000000000000000000000000000 P ce79b12353984ed19f0e2c394abb4724 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce79b12353984ed19f0e2c394abb4724; no voters:
I20260808 11:43:09.852957 26961 leader_election.cc:288] T 00000000000000000000000000000000 P ce79b12353984ed19f0e2c394abb4724 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:43:09.853061 26975 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce79b12353984ed19f0e2c394abb4724 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:43:09.854097 26975 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce79b12353984ed19f0e2c394abb4724 [term 1 LEADER]: Becoming Leader. State: Replica: ce79b12353984ed19f0e2c394abb4724, State: Running, Role: LEADER
I20260808 11:43:09.854729 26975 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce79b12353984ed19f0e2c394abb4724 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce79b12353984ed19f0e2c394abb4724" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 32805 } }
I20260808 11:43:09.855357 26961 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce79b12353984ed19f0e2c394abb4724 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:43:09.857616 26976 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce79b12353984ed19f0e2c394abb4724 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce79b12353984ed19f0e2c394abb4724" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce79b12353984ed19f0e2c394abb4724" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 32805 } } }
I20260808 11:43:09.858070 26976 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce79b12353984ed19f0e2c394abb4724 [sys.catalog]: This master's current role is: LEADER
I20260808 11:43:09.860042 26977 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce79b12353984ed19f0e2c394abb4724 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce79b12353984ed19f0e2c394abb4724. Latest consensus state: current_term: 1 leader_uuid: "ce79b12353984ed19f0e2c394abb4724" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce79b12353984ed19f0e2c394abb4724" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 32805 } } }
I20260808 11:43:09.863541 26977 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce79b12353984ed19f0e2c394abb4724 [sys.catalog]: This master's current role is: LEADER
I20260808 11:43:09.866842 26981 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:43:09.866910 26966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:09.848795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34551 (local address 127.24.76.126:32805)
0808 11:43:09.848988 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:09.848994 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:09.849481 (+ 487us) server_negotiation.cc:407] Connection header received
0808 11:43:09.849632 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:09.849637 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:09.849712 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:09.849818 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:09.851354 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:09.852437 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:09.853912 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:09.854525 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:09.854608 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:09.857935 (+ 3327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:09.857942 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:09.857956 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:09.857996 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:09.861993 (+ 3997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:09.863901 (+ 1908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:09.863908 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:09.863912 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:09.864102 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:09.864818 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:09.864839 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:09.864843 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:09.865362 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:09.865677 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:09.866487 (+ 810us) server_negotiation.cc:299] Negotiation successful
0808 11:43:09.866642 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 11:43:09.872337 26981 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:43:09.888446 26981 catalog_manager.cc:1347] Generated new cluster ID: 7581c400ecbb46d88993d7063798401f
I20260808 11:43:09.888592 26981 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:43:09.922820 26981 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:43:09.924640 26981 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:43:09.936026 26981 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce79b12353984ed19f0e2c394abb4724: Generated new TSK 0
I20260808 11:43:09.937124 26981 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:43:09.948314 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 26849
I20260808 11:43:09.949756 26953 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:43:10.042229 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 26849
W20260808 11:43:10.065819 26828 connection.cc:535] client connection to 127.24.76.126:32805 recv error: Network error: failed to read from TLS socket (remote: 127.24.76.126:32805): Connection reset by peer (error 104)
2026-08-08T11:43:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2215 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:43:10 dist-test-slave-tc2s krb5kdc[27007](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:43:10 dist-test-slave-tc2s krb5kdc[27007](info): set up 2 sockets
Aug 08 11:43:10 dist-test-slave-tc2s krb5kdc[27007](info): commencing operation
krb5kdc: starting...
W20260808 11:43:11.276667 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.006s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:43:11 dist-test-slave-tc2s krb5kdc[27007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:43:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:43:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
I20260808 11:43:11.585287 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:42453
--webserver_interface=127.24.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:46087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:42453
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:43:11.920326 27033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:43:11.920615 27033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:43:11.920744 27033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:43:11.927912 27033 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:43:11.928020 27033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:43:11.928097 27033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:43:11.928179 27033 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:43:11.928236 27033 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:43:11.937911 27033 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:46087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:42453
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:42453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:43:11.939158 27033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:43:11.940448 27033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:43:11.951725 27038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:11.951763 27039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:11.951687 27041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:43:11.952883 27033 server_base.cc:1034] running on GCE node
I20260808 11:43:11.953600 27033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:43:11.954923 27033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:43:11.956128 27033 hybrid_clock.cc:648] HybridClock initialized: now 1786189391956091 us; error 24 us; skew 500 ppm
I20260808 11:43:11.956436 27033 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:43:11 dist-test-slave-tc2s krb5kdc[27007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189391, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:43:11.959415 27033 init.cc:377] Logged in from keytab as kudu/127.24.76.126@KRBTEST.COM (short username kudu)
I20260808 11:43:11.960875 27033 webserver.cc:460] Webserver started at http://127.24.76.126:40561/ using document root <none> and password file <none>
I20260808 11:43:11.961504 27033 fs_manager.cc:359] Metadata directory not provided
I20260808 11:43:11.961614 27033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:43:11.961956 27033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:43:11.964838 27033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "4e4bd67809c34c84a319d781606957e2"
format_stamp: "Formatted at 2026-08-08 11:43:11 on dist-test-slave-tc2s"
server_key: "97e1a7cf17026a687ea4ce7cad7d8f22"
server_key_iv: "3d3b44bc80f14f7b067f142701a564d0"
server_key_version: "encryptionkey@0"
I20260808 11:43:11.965577 27033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "4e4bd67809c34c84a319d781606957e2"
format_stamp: "Formatted at 2026-08-08 11:43:11 on dist-test-slave-tc2s"
server_key: "97e1a7cf17026a687ea4ce7cad7d8f22"
server_key_iv: "3d3b44bc80f14f7b067f142701a564d0"
server_key_version: "encryptionkey@0"
I20260808 11:43:11.970646 27033 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 11:43:11.973918 27053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:11.975178 27033 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:43:11.975329 27033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "4e4bd67809c34c84a319d781606957e2"
format_stamp: "Formatted at 2026-08-08 11:43:11 on dist-test-slave-tc2s"
server_key: "97e1a7cf17026a687ea4ce7cad7d8f22"
server_key_iv: "3d3b44bc80f14f7b067f142701a564d0"
server_key_version: "encryptionkey@0"
I20260808 11:43:11.975514 27033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:43:11.986596 27033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:43:11.990568 27033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:43:11.990818 27033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:43:12.007926 27033 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:42453
I20260808 11:43:12.007963 27135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:42453 every 8 connection(s)
I20260808 11:43:12.009224 27033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:43:12.012126 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 27033
I20260808 11:43:12.012308 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:43:12.012842 24881 external_mini_cluster.cc:1428] Setting key bdcb8de53d284042548ee4568757a508
I20260808 11:43:12.013091 27138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:12.026278 27138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e4bd67809c34c84a319d781606957e2: Bootstrap starting.
I20260808 11:43:12.026355 27144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:12.014589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32977 (local address 127.24.76.126:42453)
0808 11:43:12.015308 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:12.015319 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:12.015360 (+ 41us) server_negotiation.cc:407] Connection header received
0808 11:43:12.017477 (+ 2117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:12.017501 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:12.017954 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:12.018297 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:12.018854 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:12.020169 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:12.021981 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:12.022657 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:12.022792 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:12.024115 (+ 1323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:12.024122 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:12.024134 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:43:12.024658 (+ 524us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:12.024731 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:43:12.024762 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:12.024887 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:12.025135 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 11:43:12.025301 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":341,"threads_started":1}
I20260808 11:43:12.029352 27138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e4bd67809c34c84a319d781606957e2: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:12.029987 24881 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:43:12.030484 27138 log.cc:824] T 00000000000000000000000000000000 P 4e4bd67809c34c84a319d781606957e2: Log is configured to *not* fsync() on all Append() calls
I20260808 11:43:12.033679 27138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e4bd67809c34c84a319d781606957e2: No bootstrap required, opened a new log
I20260808 11:43:12.041329 27144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:12.034848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49883 (local address 127.24.76.126:42453)
0808 11:43:12.035026 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:12.035031 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:12.035411 (+ 380us) server_negotiation.cc:407] Connection header received
0808 11:43:12.035552 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:12.035557 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:12.035632 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:12.035739 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:12.036141 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:12.037274 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:12.038559 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:12.039257 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:12.039403 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:12.040533 (+ 1130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:12.040540 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:12.040545 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:43:12.040568 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:12.040617 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:43:12.040638 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:12.040756 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:12.040942 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 11:43:12.041000 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 11:43:12.042811 27138 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e4bd67809c34c84a319d781606957e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e4bd67809c34c84a319d781606957e2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 42453 } }
I20260808 11:43:12.043144 27138 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e4bd67809c34c84a319d781606957e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:12.043226 27138 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e4bd67809c34c84a319d781606957e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e4bd67809c34c84a319d781606957e2, State: Initialized, Role: FOLLOWER
I20260808 11:43:12.043694 27138 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e4bd67809c34c84a319d781606957e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e4bd67809c34c84a319d781606957e2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 42453 } }
I20260808 11:43:12.043881 27138 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e4bd67809c34c84a319d781606957e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:43:12.043980 27138 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e4bd67809c34c84a319d781606957e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:43:12.044122 27138 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e4bd67809c34c84a319d781606957e2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:12.046250 27138 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e4bd67809c34c84a319d781606957e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e4bd67809c34c84a319d781606957e2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 42453 } }
I20260808 11:43:12.046701 27138 leader_election.cc:302] T 00000000000000000000000000000000 P 4e4bd67809c34c84a319d781606957e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e4bd67809c34c84a319d781606957e2; no voters:
I20260808 11:43:12.047219 27138 leader_election.cc:288] T 00000000000000000000000000000000 P 4e4bd67809c34c84a319d781606957e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:43:12.047328 27156 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e4bd67809c34c84a319d781606957e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:43:12.047981 27156 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e4bd67809c34c84a319d781606957e2 [term 1 LEADER]: Becoming Leader. State: Replica: 4e4bd67809c34c84a319d781606957e2, State: Running, Role: LEADER
I20260808 11:43:12.048485 27156 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e4bd67809c34c84a319d781606957e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e4bd67809c34c84a319d781606957e2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 42453 } }
I20260808 11:43:12.049108 27138 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e4bd67809c34c84a319d781606957e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:43:12.051030 27158 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e4bd67809c34c84a319d781606957e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e4bd67809c34c84a319d781606957e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e4bd67809c34c84a319d781606957e2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 42453 } } }
I20260808 11:43:12.051419 27158 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e4bd67809c34c84a319d781606957e2 [sys.catalog]: This master's current role is: LEADER
I20260808 11:43:12.051515 27159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e4bd67809c34c84a319d781606957e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e4bd67809c34c84a319d781606957e2. Latest consensus state: current_term: 1 leader_uuid: "4e4bd67809c34c84a319d781606957e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e4bd67809c34c84a319d781606957e2" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 42453 } } }
I20260808 11:43:12.051780 27159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e4bd67809c34c84a319d781606957e2 [sys.catalog]: This master's current role is: LEADER
I20260808 11:43:12.054044 27162 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:43:12.059592 27162 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:43:12.067423 27162 catalog_manager.cc:1347] Generated new cluster ID: ab71b36ec5534387aec04088350d45dc
I20260808 11:43:12.067541 27162 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:43:12.080439 27162 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:43:12.081653 27162 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:43:12.087872 27162 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e4bd67809c34c84a319d781606957e2: Generated new TSK 0
I20260808 11:43:12.088562 27162 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:43:12.094420 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 27033
W20260808 11:43:12.095449 27128 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:32977
W20260808 11:43:12.096124 24881 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 27033: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:43:12.096277 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 27033
W20260808 11:43:12.119786 27004 connection.cc:535] client connection to 127.24.76.126:42453 recv error: Network error: failed to read from TLS socket (remote: 127.24.76.126:42453): Connection reset by peer (error 104)
2026-08-08T11:43:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2054 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:43:12 dist-test-slave-tc2s krb5kdc[27186](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:43:12 dist-test-slave-tc2s krb5kdc[27186](info): set up 2 sockets
Aug 08 11:43:12 dist-test-slave-tc2s krb5kdc[27186](info): commencing operation
krb5kdc: starting...
W20260808 11:43:13.329320 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.001s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:43:13 dist-test-slave-tc2s krb5kdc[27186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:43:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:43:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
I20260808 11:43:13.646875 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:41221
--webserver_interface=127.24.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:46517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:41221
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:43:13.985260 27206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:43:13.985558 27206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:43:13.985702 27206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:43:13.992815 27206 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:43:13.992938 27206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:43:13.993014 27206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:43:13.993098 27206 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:43:13.993157 27206 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:43:14.002645 27206 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:46517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:41221
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:41221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:43:14.003847 27206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:43:14.005152 27206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:43:14.016578 27216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:14.016659 27215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:14.016626 27220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:43:14.017484 27206 server_base.cc:1034] running on GCE node
I20260808 11:43:14.018260 27206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:43:14.019495 27206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:43:14.020689 27206 hybrid_clock.cc:648] HybridClock initialized: now 1786189394020644 us; error 37 us; skew 500 ppm
I20260808 11:43:14.021028 27206 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:43:14 dist-test-slave-tc2s krb5kdc[27186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189394, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:43:14.024044 27206 init.cc:377] Logged in from keytab as kudu/127.24.76.126@KRBTEST.COM (short username kudu)
I20260808 11:43:14.025427 27206 webserver.cc:460] Webserver started at http://127.24.76.126:33609/ using document root <none> and password file <none>
I20260808 11:43:14.026139 27206 fs_manager.cc:359] Metadata directory not provided
I20260808 11:43:14.026276 27206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:43:14.026577 27206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:43:14.029381 27206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "8cdca9bbb3854d899c4cefdde02b514c"
format_stamp: "Formatted at 2026-08-08 11:43:14 on dist-test-slave-tc2s"
server_key: "3b1d049b34b6301b02d184c6649307af"
server_key_iv: "366fe866c6997419e3bf513f04f9ea9d"
server_key_version: "encryptionkey@0"
I20260808 11:43:14.030157 27206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "8cdca9bbb3854d899c4cefdde02b514c"
format_stamp: "Formatted at 2026-08-08 11:43:14 on dist-test-slave-tc2s"
server_key: "3b1d049b34b6301b02d184c6649307af"
server_key_iv: "366fe866c6997419e3bf513f04f9ea9d"
server_key_version: "encryptionkey@0"
I20260808 11:43:14.035099 27206 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 11:43:14.038450 27232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:14.039726 27206 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:43:14.039883 27206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "8cdca9bbb3854d899c4cefdde02b514c"
format_stamp: "Formatted at 2026-08-08 11:43:14 on dist-test-slave-tc2s"
server_key: "3b1d049b34b6301b02d184c6649307af"
server_key_iv: "366fe866c6997419e3bf513f04f9ea9d"
server_key_version: "encryptionkey@0"
I20260808 11:43:14.040057 27206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:43:14.045594 27206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:43:14.049283 27206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:43:14.049544 27206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:43:14.066606 27206 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:41221
I20260808 11:43:14.066645 27314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:41221 every 8 connection(s)
I20260808 11:43:14.067899 27206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:43:14.071434 27315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:14.077282 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 27206
I20260808 11:43:14.077436 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:43:14.077920 24881 external_mini_cluster.cc:1428] Setting key 11372eb11e9c1a3128fbaeec4eb92d85
I20260808 11:43:14.082614 27315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cdca9bbb3854d899c4cefdde02b514c: Bootstrap starting.
I20260808 11:43:14.086506 27315 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8cdca9bbb3854d899c4cefdde02b514c: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:14.086648 27321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:14.079668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42203 (local address 127.24.76.126:41221)
0808 11:43:14.080597 (+ 929us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:14.080613 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:14.080676 (+ 63us) server_negotiation.cc:407] Connection header received
0808 11:43:14.083001 (+ 2325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:14.083029 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:14.083436 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:14.083840 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:14.084831 (+ 991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:14.084839 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:14.084859 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:43:14.084878 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:14.084946 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:43:14.084977 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:14.085177 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:14.085307 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 11:43:14.085490 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":319,"threads_started":1}
I20260808 11:43:14.087616 27315 log.cc:824] T 00000000000000000000000000000000 P 8cdca9bbb3854d899c4cefdde02b514c: Log is configured to *not* fsync() on all Append() calls
I20260808 11:43:14.090159 24881 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:43:14.090503 27315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cdca9bbb3854d899c4cefdde02b514c: No bootstrap required, opened a new log
I20260808 11:43:14.096449 27321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:14.094173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40153 (local address 127.24.76.126:41221)
0808 11:43:14.094333 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:14.094339 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:14.094705 (+ 366us) server_negotiation.cc:407] Connection header received
0808 11:43:14.094855 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:14.094859 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:14.094900 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:14.094997 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:14.095847 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:14.095852 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:14.095857 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:43:14.095877 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:14.095912 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:43:14.095926 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:14.096033 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:14.096142 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 11:43:14.096204 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 11:43:14.098281 27315 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8cdca9bbb3854d899c4cefdde02b514c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cdca9bbb3854d899c4cefdde02b514c" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41221 } }
I20260808 11:43:14.098608 27315 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8cdca9bbb3854d899c4cefdde02b514c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:14.098699 27315 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8cdca9bbb3854d899c4cefdde02b514c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cdca9bbb3854d899c4cefdde02b514c, State: Initialized, Role: FOLLOWER
I20260808 11:43:14.099193 27315 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cdca9bbb3854d899c4cefdde02b514c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cdca9bbb3854d899c4cefdde02b514c" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41221 } }
I20260808 11:43:14.099411 27315 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8cdca9bbb3854d899c4cefdde02b514c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:43:14.099530 27315 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cdca9bbb3854d899c4cefdde02b514c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:43:14.099694 27315 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8cdca9bbb3854d899c4cefdde02b514c [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:14.102029 27315 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cdca9bbb3854d899c4cefdde02b514c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cdca9bbb3854d899c4cefdde02b514c" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41221 } }
I20260808 11:43:14.102483 27315 leader_election.cc:302] T 00000000000000000000000000000000 P 8cdca9bbb3854d899c4cefdde02b514c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8cdca9bbb3854d899c4cefdde02b514c; no voters:
I20260808 11:43:14.102995 27315 leader_election.cc:288] T 00000000000000000000000000000000 P 8cdca9bbb3854d899c4cefdde02b514c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:43:14.103113 27333 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cdca9bbb3854d899c4cefdde02b514c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:43:14.103706 27333 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8cdca9bbb3854d899c4cefdde02b514c [term 1 LEADER]: Becoming Leader. State: Replica: 8cdca9bbb3854d899c4cefdde02b514c, State: Running, Role: LEADER
I20260808 11:43:14.104245 27333 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cdca9bbb3854d899c4cefdde02b514c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cdca9bbb3854d899c4cefdde02b514c" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41221 } }
I20260808 11:43:14.104962 27315 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8cdca9bbb3854d899c4cefdde02b514c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:43:14.106920 27336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cdca9bbb3854d899c4cefdde02b514c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cdca9bbb3854d899c4cefdde02b514c. Latest consensus state: current_term: 1 leader_uuid: "8cdca9bbb3854d899c4cefdde02b514c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cdca9bbb3854d899c4cefdde02b514c" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41221 } } }
I20260808 11:43:14.107285 27336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cdca9bbb3854d899c4cefdde02b514c [sys.catalog]: This master's current role is: LEADER
I20260808 11:43:14.107873 27334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cdca9bbb3854d899c4cefdde02b514c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8cdca9bbb3854d899c4cefdde02b514c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cdca9bbb3854d899c4cefdde02b514c" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41221 } } }
I20260808 11:43:14.108193 27334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cdca9bbb3854d899c4cefdde02b514c [sys.catalog]: This master's current role is: LEADER
I20260808 11:43:14.108886 27344 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:43:14.113821 27344 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:43:14.122596 27344 catalog_manager.cc:1347] Generated new cluster ID: d6188146f4c546e4b48f9097ef9238ec
I20260808 11:43:14.122704 27344 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:43:14.137422 27344 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:43:14.137956 27344 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:43:14.146297 27344 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8cdca9bbb3854d899c4cefdde02b514c: Generated new TSK 0
I20260808 11:43:14.147250 27344 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:43:14.156456 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 27206
W20260808 11:43:14.157495 27307 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:42203
W20260808 11:43:14.158142 24881 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 27206: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:43:14.158298 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 27206
2026-08-08T11:43:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2063 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:43:14 dist-test-slave-tc2s krb5kdc[27370](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:43:14 dist-test-slave-tc2s krb5kdc[27370](info): set up 2 sockets
Aug 08 11:43:14 dist-test-slave-tc2s krb5kdc[27370](info): commencing operation
krb5kdc: starting...
W20260808 11:43:15.390903 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.004s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:43:15 dist-test-slave-tc2s krb5kdc[27370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:43:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:43:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
I20260808 11:43:15.708729 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:44861
--webserver_interface=127.24.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:45597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:44861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:43:16.048262 27391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:43:16.048590 27391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:43:16.048735 27391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:43:16.055847 27391 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:43:16.055979 27391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:43:16.056058 27391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:43:16.056143 27391 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:43:16.056205 27391 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:43:16.065840 27391 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:45597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:44861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:44861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:43:16.067176 27391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:43:16.068500 27391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:43:16.080061 27397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:16.080107 27398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:16.080019 27400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:43:16.083353 27391 server_base.cc:1034] running on GCE node
I20260808 11:43:16.084146 27391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:43:16.085510 27391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:43:16.086719 27391 hybrid_clock.cc:648] HybridClock initialized: now 1786189396086662 us; error 42 us; skew 500 ppm
I20260808 11:43:16.087064 27391 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:43:16 dist-test-slave-tc2s krb5kdc[27370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189396, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:43:16.094169 27391 init.cc:377] Logged in from keytab as kudu/127.24.76.126@KRBTEST.COM (short username kudu)
I20260808 11:43:16.095714 27391 webserver.cc:460] Webserver started at http://127.24.76.126:36719/ using document root <none> and password file <none>
I20260808 11:43:16.096426 27391 fs_manager.cc:359] Metadata directory not provided
I20260808 11:43:16.096551 27391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:43:16.096841 27391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:43:16.099872 27391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "1a5e4af799d844c6b40e95072423b4d9"
format_stamp: "Formatted at 2026-08-08 11:43:16 on dist-test-slave-tc2s"
server_key: "d11b8234ad2b7b5069ec7884f6283d36"
server_key_iv: "f532a28d39eb6cf90f7aad93e11f5719"
server_key_version: "encryptionkey@0"
I20260808 11:43:16.100660 27391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "1a5e4af799d844c6b40e95072423b4d9"
format_stamp: "Formatted at 2026-08-08 11:43:16 on dist-test-slave-tc2s"
server_key: "d11b8234ad2b7b5069ec7884f6283d36"
server_key_iv: "f532a28d39eb6cf90f7aad93e11f5719"
server_key_version: "encryptionkey@0"
I20260808 11:43:16.105719 27391 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 11:43:16.109122 27407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:16.110385 27391 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:43:16.110559 27391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "1a5e4af799d844c6b40e95072423b4d9"
format_stamp: "Formatted at 2026-08-08 11:43:16 on dist-test-slave-tc2s"
server_key: "d11b8234ad2b7b5069ec7884f6283d36"
server_key_iv: "f532a28d39eb6cf90f7aad93e11f5719"
server_key_version: "encryptionkey@0"
I20260808 11:43:16.110733 27391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:43:16.128116 27391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:43:16.131785 27391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:43:16.132035 27391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:43:16.149107 27391 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:44861
I20260808 11:43:16.149130 27495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:44861 every 8 connection(s)
I20260808 11:43:16.150496 27391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:43:16.154078 27496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:16.158418 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 27391
I20260808 11:43:16.158573 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:43:16.159001 24881 external_mini_cluster.cc:1428] Setting key fb31a81e8701517a43c652aedc02171c
I20260808 11:43:16.164731 27496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a5e4af799d844c6b40e95072423b4d9: Bootstrap starting.
I20260808 11:43:16.168293 27496 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a5e4af799d844c6b40e95072423b4d9: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:16.169441 27496 log.cc:824] T 00000000000000000000000000000000 P 1a5e4af799d844c6b40e95072423b4d9: Log is configured to *not* fsync() on all Append() calls
I20260808 11:43:16.172617 27496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a5e4af799d844c6b40e95072423b4d9: No bootstrap required, opened a new log
Aug 08 11:43:16 dist-test-slave-tc2s krb5kdc[27370](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:43:16.181584 27504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:16.160706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34639 (local address 127.24.76.126:44861)
0808 11:43:16.161481 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:16.161496 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:16.161537 (+ 41us) server_negotiation.cc:407] Connection header received
0808 11:43:16.163284 (+ 1747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:16.163307 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:16.163663 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:16.163953 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:16.165800 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:16.167159 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:16.168900 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:16.169624 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:16.169755 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:16.174538 (+ 4783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:16.174546 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:16.174565 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:16.174607 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:16.177642 (+ 3035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:16.178420 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:16.178427 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:16.178436 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:16.178603 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:16.178972 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:16.178975 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:16.178976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:16.179505 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:16.179730 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:16.180188 (+ 458us) server_negotiation.cc:299] Negotiation successful
0808 11:43:16.180576 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":334,"threads_started":1}
I20260808 11:43:16.181497 27496 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a5e4af799d844c6b40e95072423b4d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a5e4af799d844c6b40e95072423b4d9" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44861 } }
I20260808 11:43:16.181910 27496 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a5e4af799d844c6b40e95072423b4d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:16.182019 27496 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a5e4af799d844c6b40e95072423b4d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a5e4af799d844c6b40e95072423b4d9, State: Initialized, Role: FOLLOWER
I20260808 11:43:16.182569 27496 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a5e4af799d844c6b40e95072423b4d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a5e4af799d844c6b40e95072423b4d9" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44861 } }
I20260808 11:43:16.182785 27496 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a5e4af799d844c6b40e95072423b4d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:43:16.182927 27496 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a5e4af799d844c6b40e95072423b4d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:43:16.183131 27496 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a5e4af799d844c6b40e95072423b4d9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:16.185482 24881 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:43:16.186024 27496 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a5e4af799d844c6b40e95072423b4d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a5e4af799d844c6b40e95072423b4d9" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44861 } }
I20260808 11:43:16.186542 27496 leader_election.cc:302] T 00000000000000000000000000000000 P 1a5e4af799d844c6b40e95072423b4d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a5e4af799d844c6b40e95072423b4d9; no voters:
I20260808 11:43:16.187124 27496 leader_election.cc:288] T 00000000000000000000000000000000 P 1a5e4af799d844c6b40e95072423b4d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:43:16.187229 27507 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a5e4af799d844c6b40e95072423b4d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:43:16.188241 27507 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a5e4af799d844c6b40e95072423b4d9 [term 1 LEADER]: Becoming Leader. State: Replica: 1a5e4af799d844c6b40e95072423b4d9, State: Running, Role: LEADER
I20260808 11:43:16.188848 27507 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a5e4af799d844c6b40e95072423b4d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a5e4af799d844c6b40e95072423b4d9" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44861 } }
I20260808 11:43:16.189530 27496 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a5e4af799d844c6b40e95072423b4d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:43:16.191341 27510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a5e4af799d844c6b40e95072423b4d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a5e4af799d844c6b40e95072423b4d9. Latest consensus state: current_term: 1 leader_uuid: "1a5e4af799d844c6b40e95072423b4d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a5e4af799d844c6b40e95072423b4d9" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44861 } } }
I20260808 11:43:16.191721 27510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a5e4af799d844c6b40e95072423b4d9 [sys.catalog]: This master's current role is: LEADER
I20260808 11:43:16.192247 27509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a5e4af799d844c6b40e95072423b4d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a5e4af799d844c6b40e95072423b4d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a5e4af799d844c6b40e95072423b4d9" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 44861 } } }
I20260808 11:43:16.192561 27509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a5e4af799d844c6b40e95072423b4d9 [sys.catalog]: This master's current role is: LEADER
I20260808 11:43:16.195984 27515 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:43:16.201200 27515 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:43:16.228173 27515 catalog_manager.cc:1347] Generated new cluster ID: 8d6739abf2184f10ba010a37ba871931
I20260808 11:43:16.228302 27515 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:43:16.244848 27504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:16.216238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42611 (local address 127.24.76.126:44861)
0808 11:43:16.216412 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:16.216418 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:16.226493 (+ 10075us) server_negotiation.cc:407] Connection header received
0808 11:43:16.226698 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:16.226705 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:16.226805 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:16.226930 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:16.229512 (+ 2582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:16.230744 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:16.232691 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:16.233452 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:16.233565 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:16.237139 (+ 3574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:16.237147 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:16.237153 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:16.237202 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:16.241198 (+ 3996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:16.242154 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:16.242160 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:16.242165 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:16.242350 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:16.242834 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:16.242840 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:16.242843 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:16.243256 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:16.243500 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:16.244167 (+ 667us) server_negotiation.cc:299] Negotiation successful
0808 11:43:16.244421 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 11:43:16.251827 27515 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:43:16.253536 27515 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:43:16.260766 27515 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a5e4af799d844c6b40e95072423b4d9: Generated new TSK 0
I20260808 11:43:16.261572 27515 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:43:16.274686 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 27391
I20260808 11:43:16.275960 27491 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:43:16.367255 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 27391
2026-08-08T11:43:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2207 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:43:16 dist-test-slave-tc2s krb5kdc[27545](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:43:16 dist-test-slave-tc2s krb5kdc[27545](info): set up 2 sockets
Aug 08 11:43:16 dist-test-slave-tc2s krb5kdc[27545](info): commencing operation
krb5kdc: starting...
W20260808 11:43:17.612871 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.202s user 0.001s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:43:17 dist-test-slave-tc2s krb5kdc[27545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:43:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:43:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
I20260808 11:43:17.943116 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:43599
--webserver_interface=127.24.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:40793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:43599
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:43:18.281586 27564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:43:18.281879 27564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:43:18.282042 27564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:43:18.289149 27564 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:43:18.289270 27564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:43:18.289350 27564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:43:18.289433 27564 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:43:18.289495 27564 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:43:18.299320 27564 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:40793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:43599
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:43599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:43:18.300613 27564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:43:18.302032 27564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:43:18.313426 27574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:18.313455 27576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:18.313485 27579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:43:18.314157 27564 server_base.cc:1034] running on GCE node
I20260808 11:43:18.315155 27564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:43:18.316808 27564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:43:18.317991 27564 hybrid_clock.cc:648] HybridClock initialized: now 1786189398317933 us; error 40 us; skew 500 ppm
I20260808 11:43:18.318425 27564 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:43:18 dist-test-slave-tc2s krb5kdc[27545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189398, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:43:18.322925 27564 init.cc:377] Logged in from keytab as kudu/127.24.76.126@KRBTEST.COM (short username kudu)
I20260808 11:43:18.325976 27564 webserver.cc:460] Webserver started at http://127.24.76.126:37287/ using document root <none> and password file <none>
I20260808 11:43:18.326745 27564 fs_manager.cc:359] Metadata directory not provided
I20260808 11:43:18.326876 27564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:43:18.327127 27564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:43:18.330143 27564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "9084f040289a45d4b5fcc36612ddc489"
format_stamp: "Formatted at 2026-08-08 11:43:18 on dist-test-slave-tc2s"
server_key: "0a87f4ca6ee028085d487954137a4ccd"
server_key_iv: "71d29a301a35f10d4d357a98d4c6b869"
server_key_version: "encryptionkey@0"
I20260808 11:43:18.330896 27564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "9084f040289a45d4b5fcc36612ddc489"
format_stamp: "Formatted at 2026-08-08 11:43:18 on dist-test-slave-tc2s"
server_key: "0a87f4ca6ee028085d487954137a4ccd"
server_key_iv: "71d29a301a35f10d4d357a98d4c6b869"
server_key_version: "encryptionkey@0"
I20260808 11:43:18.335886 27564 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 11:43:18.339167 27588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:18.340399 27564 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 11:43:18.340547 27564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "9084f040289a45d4b5fcc36612ddc489"
format_stamp: "Formatted at 2026-08-08 11:43:18 on dist-test-slave-tc2s"
server_key: "0a87f4ca6ee028085d487954137a4ccd"
server_key_iv: "71d29a301a35f10d4d357a98d4c6b869"
server_key_version: "encryptionkey@0"
I20260808 11:43:18.340715 27564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:43:18.356613 27564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:43:18.360252 27564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:43:18.360519 27564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:43:18.377861 27564 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:43599
I20260808 11:43:18.377910 27678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:43599 every 8 connection(s)
I20260808 11:43:18.379184 27564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:43:18.382813 27679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:18.383989 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 27564
I20260808 11:43:18.384150 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:43:18.384570 24881 external_mini_cluster.cc:1428] Setting key 20addee044ca02227762537e395066e7
I20260808 11:43:18.396903 27679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9084f040289a45d4b5fcc36612ddc489: Bootstrap starting.
I20260808 11:43:18.398967 27683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:18.386346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56557 (local address 127.24.76.126:43599)
0808 11:43:18.387223 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:18.387238 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:18.387289 (+ 51us) server_negotiation.cc:407] Connection header received
0808 11:43:18.389125 (+ 1836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:18.389154 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:18.389642 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:18.390115 (+ 473us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:18.390620 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:18.392466 (+ 1846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:18.393834 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:18.394938 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:18.395117 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:18.396218 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:18.396229 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:18.396246 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:43:18.396938 (+ 692us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:18.397046 (+ 108us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:43:18.397080 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:18.397195 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:18.397380 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 11:43:18.397641 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":326,"threads_started":1}
I20260808 11:43:18.400446 27679 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9084f040289a45d4b5fcc36612ddc489: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:18.401518 27679 log.cc:824] T 00000000000000000000000000000000 P 9084f040289a45d4b5fcc36612ddc489: Log is configured to *not* fsync() on all Append() calls
I20260808 11:43:18.402503 24881 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:43:18.404235 27679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9084f040289a45d4b5fcc36612ddc489: No bootstrap required, opened a new log
I20260808 11:43:18.413595 27683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:18.406915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56395 (local address 127.24.76.126:43599)
0808 11:43:18.407115 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:18.407121 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:18.407535 (+ 414us) server_negotiation.cc:407] Connection header received
0808 11:43:18.407703 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:18.407706 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:18.407770 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:18.407848 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:18.408514 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:18.409246 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:18.411104 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:18.411590 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:18.411684 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:18.412794 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:18.412800 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:18.412806 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:43:18.412827 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:18.412870 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:43:18.412892 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:18.412979 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:18.413143 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 11:43:18.413224 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 11:43:18.413378 27679 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9084f040289a45d4b5fcc36612ddc489 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9084f040289a45d4b5fcc36612ddc489" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 43599 } }
I20260808 11:43:18.413928 27679 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9084f040289a45d4b5fcc36612ddc489 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:18.414058 27679 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9084f040289a45d4b5fcc36612ddc489 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9084f040289a45d4b5fcc36612ddc489, State: Initialized, Role: FOLLOWER
I20260808 11:43:18.414727 27679 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9084f040289a45d4b5fcc36612ddc489 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9084f040289a45d4b5fcc36612ddc489" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 43599 } }
I20260808 11:43:18.415007 27679 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9084f040289a45d4b5fcc36612ddc489 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:43:18.415155 27679 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9084f040289a45d4b5fcc36612ddc489 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:43:18.415333 27679 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9084f040289a45d4b5fcc36612ddc489 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:18.417462 27679 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9084f040289a45d4b5fcc36612ddc489 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9084f040289a45d4b5fcc36612ddc489" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 43599 } }
I20260808 11:43:18.417995 27679 leader_election.cc:302] T 00000000000000000000000000000000 P 9084f040289a45d4b5fcc36612ddc489 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9084f040289a45d4b5fcc36612ddc489; no voters:
I20260808 11:43:18.418545 27679 leader_election.cc:288] T 00000000000000000000000000000000 P 9084f040289a45d4b5fcc36612ddc489 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:43:18.418668 27696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9084f040289a45d4b5fcc36612ddc489 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:43:18.419986 27696 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9084f040289a45d4b5fcc36612ddc489 [term 1 LEADER]: Becoming Leader. State: Replica: 9084f040289a45d4b5fcc36612ddc489, State: Running, Role: LEADER
I20260808 11:43:18.420503 27696 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9084f040289a45d4b5fcc36612ddc489 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9084f040289a45d4b5fcc36612ddc489" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 43599 } }
I20260808 11:43:18.420729 27679 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9084f040289a45d4b5fcc36612ddc489 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:43:18.423249 27699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9084f040289a45d4b5fcc36612ddc489 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9084f040289a45d4b5fcc36612ddc489. Latest consensus state: current_term: 1 leader_uuid: "9084f040289a45d4b5fcc36612ddc489" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9084f040289a45d4b5fcc36612ddc489" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 43599 } } }
I20260808 11:43:18.423398 27697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9084f040289a45d4b5fcc36612ddc489 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9084f040289a45d4b5fcc36612ddc489" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9084f040289a45d4b5fcc36612ddc489" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 43599 } } }
I20260808 11:43:18.423646 27699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9084f040289a45d4b5fcc36612ddc489 [sys.catalog]: This master's current role is: LEADER
I20260808 11:43:18.423663 27697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9084f040289a45d4b5fcc36612ddc489 [sys.catalog]: This master's current role is: LEADER
I20260808 11:43:18.425531 27711 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:43:18.430544 27711 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:43:18.438927 27711 catalog_manager.cc:1347] Generated new cluster ID: e05c58a83540463795843b438efb9bfb
I20260808 11:43:18.439071 27711 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:43:18.451359 27711 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:43:18.452571 27711 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:43:18.459523 27711 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9084f040289a45d4b5fcc36612ddc489: Generated new TSK 0
I20260808 11:43:18.460490 27711 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:43:18.471402 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 27564
W20260808 11:43:18.472473 27670 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:56557
W20260808 11:43:18.473104 24881 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 27564: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:43:18.473232 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 27564
2026-08-08T11:43:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2108 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:43:18 dist-test-slave-tc2s krb5kdc[27732](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:43:18 dist-test-slave-tc2s krb5kdc[27732](info): set up 2 sockets
Aug 08 11:43:18 dist-test-slave-tc2s krb5kdc[27732](info): commencing operation
krb5kdc: starting...
W20260808 11:43:19.720374 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.005s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:43:19 dist-test-slave-tc2s krb5kdc[27732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:43:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:43:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
I20260808 11:43:20.055465 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:36059
--webserver_interface=127.24.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:38429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:36059
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:43:20.398949 27749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:43:20.399230 27749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:43:20.399359 27749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:43:20.406498 27749 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:43:20.406610 27749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:43:20.406683 27749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:43:20.406759 27749 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:43:20.406812 27749 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:43:20.416386 27749 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:38429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:36059
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:36059
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:43:20.417834 27749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:43:20.419500 27749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:43:20.431391 27758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:20.431403 27759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:43:20.431773 27749 server_base.cc:1034] running on GCE node
W20260808 11:43:20.431532 27761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:43:20.432809 27749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:43:20.434175 27749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:43:20.435426 27749 hybrid_clock.cc:648] HybridClock initialized: now 1786189400435373 us; error 42 us; skew 500 ppm
I20260808 11:43:20.435765 27749 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:43:20 dist-test-slave-tc2s krb5kdc[27732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189400, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:43:20.438822 27749 init.cc:377] Logged in from keytab as kudu/127.24.76.126@KRBTEST.COM (short username kudu)
I20260808 11:43:20.440167 27749 webserver.cc:460] Webserver started at http://127.24.76.126:46791/ using document root <none> and password file <none>
I20260808 11:43:20.440829 27749 fs_manager.cc:359] Metadata directory not provided
I20260808 11:43:20.440956 27749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:43:20.441246 27749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:43:20.444185 27749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "e323c39fac0f4bd6ba980e4fb9072793"
format_stamp: "Formatted at 2026-08-08 11:43:20 on dist-test-slave-tc2s"
server_key: "352755ac6754cbf804c796bea585b877"
server_key_iv: "3aa09f825f5ad72b9d032e8b98231ba1"
server_key_version: "encryptionkey@0"
I20260808 11:43:20.444939 27749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "e323c39fac0f4bd6ba980e4fb9072793"
format_stamp: "Formatted at 2026-08-08 11:43:20 on dist-test-slave-tc2s"
server_key: "352755ac6754cbf804c796bea585b877"
server_key_iv: "3aa09f825f5ad72b9d032e8b98231ba1"
server_key_version: "encryptionkey@0"
I20260808 11:43:20.449935 27749 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260808 11:43:20.453197 27770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:20.454521 27749 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 11:43:20.454676 27749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "e323c39fac0f4bd6ba980e4fb9072793"
format_stamp: "Formatted at 2026-08-08 11:43:20 on dist-test-slave-tc2s"
server_key: "352755ac6754cbf804c796bea585b877"
server_key_iv: "3aa09f825f5ad72b9d032e8b98231ba1"
server_key_version: "encryptionkey@0"
I20260808 11:43:20.454854 27749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:43:20.460342 27749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:43:20.464461 27749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:43:20.464737 27749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:43:20.482038 27749 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:36059
I20260808 11:43:20.482076 27848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:36059 every 8 connection(s)
I20260808 11:43:20.483381 27749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:43:20.486027 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 27749
I20260808 11:43:20.486192 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:43:20.486626 24881 external_mini_cluster.cc:1428] Setting key 1f0d7f864d7ee1d22eedbc948faf925d
I20260808 11:43:20.487340 27849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:20.494045 27854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:20.488309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38147 (local address 127.24.76.126:36059)
0808 11:43:20.489065 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:20.489076 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:20.489116 (+ 40us) server_negotiation.cc:407] Connection header received
0808 11:43:20.490668 (+ 1552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:20.490690 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:20.491018 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:20.491348 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:20.492428 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:20.492435 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:20.492448 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:43:20.492490 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:20.492563 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:43:20.492582 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:20.492746 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:20.492895 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 11:43:20.493074 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":323,"threads_started":1}
I20260808 11:43:20.497555 24881 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:43:20.499959 27849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e323c39fac0f4bd6ba980e4fb9072793: Bootstrap starting.
I20260808 11:43:20.503569 27849 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e323c39fac0f4bd6ba980e4fb9072793: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:20.504347 27854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:20.501824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50683 (local address 127.24.76.126:36059)
0808 11:43:20.502004 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:20.502008 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:20.502548 (+ 540us) server_negotiation.cc:407] Connection header received
0808 11:43:20.502723 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:20.502726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:20.502769 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:20.502844 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:20.503745 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:20.503749 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:20.503752 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 11:43:20.503766 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:20.503801 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 11:43:20.503814 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:20.503902 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:20.504056 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 11:43:20.504101 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 11:43:20.504808 27849 log.cc:824] T 00000000000000000000000000000000 P e323c39fac0f4bd6ba980e4fb9072793: Log is configured to *not* fsync() on all Append() calls
I20260808 11:43:20.507373 27849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e323c39fac0f4bd6ba980e4fb9072793: No bootstrap required, opened a new log
I20260808 11:43:20.514206 27849 raft_consensus.cc:348] T 00000000000000000000000000000000 P e323c39fac0f4bd6ba980e4fb9072793 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e323c39fac0f4bd6ba980e4fb9072793" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 36059 } }
I20260808 11:43:20.514555 27849 raft_consensus.cc:374] T 00000000000000000000000000000000 P e323c39fac0f4bd6ba980e4fb9072793 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 11:43:20.514648 27849 raft_consensus.cc:729] T 00000000000000000000000000000000 P e323c39fac0f4bd6ba980e4fb9072793 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e323c39fac0f4bd6ba980e4fb9072793, State: Initialized, Role: FOLLOWER
I20260808 11:43:20.515162 27849 consensus_queue.cc:260] T 00000000000000000000000000000000 P e323c39fac0f4bd6ba980e4fb9072793 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e323c39fac0f4bd6ba980e4fb9072793" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 36059 } }
I20260808 11:43:20.515385 27849 raft_consensus.cc:388] T 00000000000000000000000000000000 P e323c39fac0f4bd6ba980e4fb9072793 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 11:43:20.515511 27849 raft_consensus.cc:482] T 00000000000000000000000000000000 P e323c39fac0f4bd6ba980e4fb9072793 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 11:43:20.515661 27849 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e323c39fac0f4bd6ba980e4fb9072793 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:20.517791 27849 raft_consensus.cc:504] T 00000000000000000000000000000000 P e323c39fac0f4bd6ba980e4fb9072793 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e323c39fac0f4bd6ba980e4fb9072793" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 36059 } }
I20260808 11:43:20.518261 27849 leader_election.cc:302] T 00000000000000000000000000000000 P e323c39fac0f4bd6ba980e4fb9072793 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e323c39fac0f4bd6ba980e4fb9072793; no voters:
I20260808 11:43:20.518782 27849 leader_election.cc:288] T 00000000000000000000000000000000 P e323c39fac0f4bd6ba980e4fb9072793 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 11:43:20.518899 27865 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e323c39fac0f4bd6ba980e4fb9072793 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:43:20.519560 27865 raft_consensus.cc:686] T 00000000000000000000000000000000 P e323c39fac0f4bd6ba980e4fb9072793 [term 1 LEADER]: Becoming Leader. State: Replica: e323c39fac0f4bd6ba980e4fb9072793, State: Running, Role: LEADER
I20260808 11:43:20.520051 27865 consensus_queue.cc:237] T 00000000000000000000000000000000 P e323c39fac0f4bd6ba980e4fb9072793 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e323c39fac0f4bd6ba980e4fb9072793" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 36059 } }
I20260808 11:43:20.520663 27849 sys_catalog.cc:564] T 00000000000000000000000000000000 P e323c39fac0f4bd6ba980e4fb9072793 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:43:20.522589 27867 sys_catalog.cc:455] T 00000000000000000000000000000000 P e323c39fac0f4bd6ba980e4fb9072793 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e323c39fac0f4bd6ba980e4fb9072793" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e323c39fac0f4bd6ba980e4fb9072793" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 36059 } } }
I20260808 11:43:20.522620 27869 sys_catalog.cc:455] T 00000000000000000000000000000000 P e323c39fac0f4bd6ba980e4fb9072793 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e323c39fac0f4bd6ba980e4fb9072793. Latest consensus state: current_term: 1 leader_uuid: "e323c39fac0f4bd6ba980e4fb9072793" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e323c39fac0f4bd6ba980e4fb9072793" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 36059 } } }
I20260808 11:43:20.523049 27867 sys_catalog.cc:458] T 00000000000000000000000000000000 P e323c39fac0f4bd6ba980e4fb9072793 [sys.catalog]: This master's current role is: LEADER
I20260808 11:43:20.523049 27869 sys_catalog.cc:458] T 00000000000000000000000000000000 P e323c39fac0f4bd6ba980e4fb9072793 [sys.catalog]: This master's current role is: LEADER
I20260808 11:43:20.525434 27873 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 11:43:20.531101 27873 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:43:20.540799 27873 catalog_manager.cc:1347] Generated new cluster ID: 3ba95ba5765447c09f9ccb1fe09e6b6a
I20260808 11:43:20.540921 27873 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:43:20.557790 27873 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:43:20.558312 27873 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:43:20.574795 27873 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e323c39fac0f4bd6ba980e4fb9072793: Generated new TSK 0
I20260808 11:43:20.575817 27873 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:43:20.581436 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 27749
W20260808 11:43:20.582592 27842 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:38147
W20260808 11:43:20.583320 24881 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 27749: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 11:43:20.583470 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 27749
2026-08-08T11:43:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2109 ms)
[----------] 18 tests from AuthTokenIssuingTest (37797 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:43:20 dist-test-slave-tc2s krb5kdc[27901](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:43:20 dist-test-slave-tc2s krb5kdc[27901](info): set up 2 sockets
Aug 08 11:43:20 dist-test-slave-tc2s krb5kdc[27901](info): commencing operation
krb5kdc: starting...
W20260808 11:43:21.831630 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.001s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:43:22 dist-test-slave-tc2s krb5kdc[27901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:43:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:43:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
I20260808 11:43:22.177664 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:39265
--webserver_interface=127.24.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:38121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:39265,127.24.76.125:37191,127.24.76.124:35599
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:43:22.522581 27924 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 11:43:22.522912 27924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:43:22.523017 27924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:43:22.523139 27924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:43:22.530227 27924 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:43:22.530315 27924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:43:22.530411 27924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:43:22.530475 27924 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:43:22.530568 27924 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:43:22.540216 27924 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:38121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:39265,127.24.76.125:37191,127.24.76.124:35599
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:39265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:43:22.541522 27924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:43:22.542846 27924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:43:22.554378 27935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:22.554416 27939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:22.554337 27933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:43:22.555197 27924 server_base.cc:1034] running on GCE node
I20260808 11:43:22.555925 27924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:43:22.557294 27924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:43:22.558516 27924 hybrid_clock.cc:648] HybridClock initialized: now 1786189402558450 us; error 34 us; skew 500 ppm
I20260808 11:43:22.558830 27924 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:43:22 dist-test-slave-tc2s krb5kdc[27901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189402, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:43:22.565757 27924 init.cc:377] Logged in from keytab as kudu/127.24.76.126@KRBTEST.COM (short username kudu)
I20260808 11:43:22.567215 27924 webserver.cc:460] Webserver started at http://127.24.76.126:44163/ using document root <none> and password file <none>
I20260808 11:43:22.567930 27924 fs_manager.cc:359] Metadata directory not provided
I20260808 11:43:22.568045 27924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:43:22.568375 27924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:43:22.571444 27924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "be0704aff1c74ec8b62f7040b26dcace"
format_stamp: "Formatted at 2026-08-08 11:43:22 on dist-test-slave-tc2s"
server_key: "dbcf5b7e5049de73e88f4f6813b77362"
server_key_iv: "de91443af3ae5e64ece3f2bd9cfa36d4"
server_key_version: "encryptionkey@0"
I20260808 11:43:22.572319 27924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "be0704aff1c74ec8b62f7040b26dcace"
format_stamp: "Formatted at 2026-08-08 11:43:22 on dist-test-slave-tc2s"
server_key: "dbcf5b7e5049de73e88f4f6813b77362"
server_key_iv: "de91443af3ae5e64ece3f2bd9cfa36d4"
server_key_version: "encryptionkey@0"
I20260808 11:43:22.577373 27924 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 11:43:22.580744 27949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:22.582041 27924 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:43:22.582201 27924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "be0704aff1c74ec8b62f7040b26dcace"
format_stamp: "Formatted at 2026-08-08 11:43:22 on dist-test-slave-tc2s"
server_key: "dbcf5b7e5049de73e88f4f6813b77362"
server_key_iv: "de91443af3ae5e64ece3f2bd9cfa36d4"
server_key_version: "encryptionkey@0"
I20260808 11:43:22.582396 27924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:43:22.600605 27924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:43:22.604475 27924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:43:22.604727 27924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:43:22.621625 27924 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:39265
I20260808 11:43:22.621650 28035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:39265 every 8 connection(s)
I20260808 11:43:22.622927 27924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:43:22.626618 28037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:22.629683 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 27924
I20260808 11:43:22.629863 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:43:22.630540 24881 external_mini_cluster.cc:1428] Setting key f1e571547a63f459c2a56542399d5948
I20260808 11:43:22.633165 28037 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39265 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:43:22 dist-test-slave-tc2s krb5kdc[27901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189402, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:43:22.655757 28039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:22.636256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:39265 (local address 127.0.0.1:50294)
0808 11:43:22.637069 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:22.637116 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:22.639154 (+ 2038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:22.639936 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:22.639942 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:22.640655 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:22.640996 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:22.641014 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:22.642606 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:22.642610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:22.643661 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:22.643675 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:22.644585 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:22.644593 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:22.644767 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:22.645458 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:22.645489 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:22.648068 (+ 2579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:22.651919 (+ 3851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:22.651930 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:22.651941 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:22.652328 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:22.652770 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:22.652773 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:22.652775 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:22.652937 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:22.653844 (+ 907us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:22.653850 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:22.654096 (+ 246us) client_negotiation.cc:769] Sending connection context
0808 11:43:22.654316 (+ 220us) client_negotiation.cc:240] Negotiation successful
0808 11:43:22.654723 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":329,"threads_started":1}
I20260808 11:43:22.655812 28040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:22.636257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50294 (local address 127.24.76.126:39265)
0808 11:43:22.637934 (+ 1677us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:22.637942 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:22.638011 (+ 69us) server_negotiation.cc:407] Connection header received
0808 11:43:22.639497 (+ 1486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:22.639503 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:22.639718 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:22.639904 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:22.641195 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:22.642459 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:22.643799 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:22.644351 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:22.644515 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:22.648326 (+ 3811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:22.648336 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:22.648352 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:22.648394 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:22.651772 (+ 3378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:22.652463 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:22.652472 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:22.652475 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:22.652666 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:22.653038 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:22.653043 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:22.653045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:22.653708 (+ 663us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:22.653987 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:22.654481 (+ 494us) server_negotiation.cc:299] Negotiation successful
0808 11:43:22.654892 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":124,"server-negotiator.queue_time_us":1394,"thread_start_us":1191,"threads_started":1}
I20260808 11:43:22.658512 28037 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.76.125" port: 37191 } has no permanent_uuid. Determining permanent_uuid...
I20260808 11:43:22.659606 28039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:22.659199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:37191 (local address 127.0.0.1:33360)
0808 11:43:22.659384 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:22.659501 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.125:37191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 11:43:22.659924 27953 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.76.125:37191: connect: Connection refused (error 111)
I20260808 11:43:22.661509 28039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:22.661159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:37191 (local address 127.0.0.1:33364)
0808 11:43:22.661312 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:22.661381 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.125:37191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 11:43:22.661792 28037 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.125:37191: Network error: Client connection negotiation failed: client connection to 127.24.76.125:37191: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.24.76.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.125.keytab.
Entry for principal kudu/127.24.76.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.125.keytab.
I20260808 11:43:22.707253 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.125:37191
--webserver_interface=127.24.76.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.125.keytab
--principal=kudu/127.24.76.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:38121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:39265,127.24.76.125:37191,127.24.76.124:35599
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:43:22.711128 28037 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.125" port: 37191 } attempt: 1
I20260808 11:43:22.712263 28039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:22.711790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:37191 (local address 127.0.0.1:33376)
0808 11:43:22.711992 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:22.712119 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.125:37191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 11:43:22.713302 28039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:22.712952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:37191 (local address 127.0.0.1:33384)
0808 11:43:22.713101 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:22.713169 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.125:37191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 11:43:22.713510 28037 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.125:37191: Network error: Client connection negotiation failed: client connection to 127.24.76.125:37191: connect: Connection refused (error 111)
I20260808 11:43:22.781797 28037 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.125" port: 37191 } attempt: 2
I20260808 11:43:22.783205 28039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:22.782602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:37191 (local address 127.0.0.1:33394)
0808 11:43:22.782893 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:22.783023 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.125:37191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260808 11:43:22.784309 28039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:22.783882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:37191 (local address 127.0.0.1:33398)
0808 11:43:22.784037 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:22.784128 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.125:37191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 11:43:22.784512 28037 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.125:37191: Network error: Client connection negotiation failed: client connection to 127.24.76.125:37191: connect: Connection refused (error 111)
I20260808 11:43:22.875806 28037 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.125" port: 37191 } attempt: 3
I20260808 11:43:22.877125 28039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:22.876598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:37191 (local address 127.0.0.1:33406)
0808 11:43:22.876838 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:22.876956 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.125:37191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260808 11:43:22.878535 28039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:22.878010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:37191 (local address 127.0.0.1:33416)
0808 11:43:22.878217 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:22.878337 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.125:37191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260808 11:43:22.878741 28037 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.125:37191: Network error: Client connection negotiation failed: client connection to 127.24.76.125:37191: connect: Connection refused (error 111)
I20260808 11:43:23.022017 28037 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.125" port: 37191 } attempt: 4
I20260808 11:43:23.023362 28039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.022728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:37191 (local address 127.0.0.1:33418)
0808 11:43:23.023040 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:23.023171 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.125:37191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167}
I20260808 11:43:23.024528 28039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.024077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:37191 (local address 127.0.0.1:33422)
0808 11:43:23.024242 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:23.024358 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.125:37191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 11:43:23.024775 28037 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.125:37191: Network error: Client connection negotiation failed: client connection to 127.24.76.125:37191: connect: Connection refused (error 111)
W20260808 11:43:23.063406 28046 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 11:43:23.063752 28046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:43:23.063874 28046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:43:23.064007 28046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:43:23.071508 28046 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:43:23.071599 28046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:43:23.071669 28046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:43:23.071739 28046 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:43:23.071832 28046 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:43:23.081911 28046 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:38121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:39265,127.24.76.125:37191,127.24.76.124:35599
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.125.keytab
--principal=kudu/127.24.76.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.125:37191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.76.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:43:23.083268 28046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:43:23.084606 28046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:43:23.096275 28054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:23.096211 28056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:23.096251 28062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:43:23.097306 28046 server_base.cc:1034] running on GCE node
I20260808 11:43:23.098171 28046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:43:23.099529 28046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:43:23.100764 28046 hybrid_clock.cc:648] HybridClock initialized: now 1786189403100688 us; error 46 us; skew 500 ppm
I20260808 11:43:23.101112 28046 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:43:23 dist-test-slave-tc2s krb5kdc[27901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189403, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:43:23.104223 28046 init.cc:377] Logged in from keytab as kudu/127.24.76.125@KRBTEST.COM (short username kudu)
I20260808 11:43:23.105659 28046 webserver.cc:460] Webserver started at http://127.24.76.125:43699/ using document root <none> and password file <none>
I20260808 11:43:23.106386 28046 fs_manager.cc:359] Metadata directory not provided
I20260808 11:43:23.106492 28046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:43:23.106794 28046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:43:23.109722 28046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-1/data/instance:
uuid: "10153b1ecada481abfb14380b258d382"
format_stamp: "Formatted at 2026-08-08 11:43:23 on dist-test-slave-tc2s"
server_key: "3c0e1fafa33a0845d682286609b6eba2"
server_key_iv: "612d6e9fa45b2f1a40d7177d17ca7ff5"
server_key_version: "encryptionkey@0"
I20260808 11:43:23.110572 28046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-1/wal/instance:
uuid: "10153b1ecada481abfb14380b258d382"
format_stamp: "Formatted at 2026-08-08 11:43:23 on dist-test-slave-tc2s"
server_key: "3c0e1fafa33a0845d682286609b6eba2"
server_key_iv: "612d6e9fa45b2f1a40d7177d17ca7ff5"
server_key_version: "encryptionkey@0"
I20260808 11:43:23.116235 28046 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260808 11:43:23.119969 28073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:23.121376 28046 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 11:43:23.121542 28046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-1/wal
uuid: "10153b1ecada481abfb14380b258d382"
format_stamp: "Formatted at 2026-08-08 11:43:23 on dist-test-slave-tc2s"
server_key: "3c0e1fafa33a0845d682286609b6eba2"
server_key_iv: "612d6e9fa45b2f1a40d7177d17ca7ff5"
server_key_version: "encryptionkey@0"
I20260808 11:43:23.121750 28046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:43:23.155892 28046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:43:23.159919 28046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:43:23.160164 28046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:43:23.177860 28046 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.125:37191
I20260808 11:43:23.177932 28156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.125:37191 every 8 connection(s)
I20260808 11:43:23.179206 28046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-1/data/info.pb
I20260808 11:43:23.181838 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 28046
I20260808 11:43:23.182054 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-1/wal/instance
I20260808 11:43:23.182613 24881 external_mini_cluster.cc:1428] Setting key 162435858910226ffca8024c239cc188
I20260808 11:43:23.183470 28161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:23.191972 28161 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39265 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:43:23 dist-test-slave-tc2s krb5kdc[27901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189403, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.125@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:43:23.214099 28165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.195055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50306 (local address 127.24.76.126:39265)
0808 11:43:23.195571 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:23.195577 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:23.196561 (+ 984us) server_negotiation.cc:407] Connection header received
0808 11:43:23.198494 (+ 1933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:23.198499 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:23.198568 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:23.198653 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:23.200655 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.201510 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.203628 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.204074 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.204175 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:23.208285 (+ 4110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:23.208290 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:23.208294 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:23.208339 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:23.210933 (+ 2594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:23.211831 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:23.211835 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:23.211837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:23.211976 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:23.212383 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:23.212386 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:23.212388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:23.212659 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:23.212765 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:23.213580 (+ 815us) server_negotiation.cc:299] Negotiation successful
0808 11:43:23.213752 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":268,"threads_started":1}
I20260808 11:43:23.215444 28166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.195628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:39265 (local address 127.0.0.1:50306)
0808 11:43:23.196380 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:23.196428 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:23.198204 (+ 1776us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:23.198913 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:23.198924 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:23.199636 (+ 712us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:23.200421 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:23.200445 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.201681 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.201688 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:23.203471 (+ 1783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:23.203490 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.204294 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.204301 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:23.204562 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:23.205321 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:23.205348 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:23.208063 (+ 2715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:23.211086 (+ 3023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:23.211097 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:23.211115 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:23.211714 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:23.212074 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:23.212078 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:23.212081 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:23.212289 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:23.212782 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:23.212790 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:23.213214 (+ 424us) client_negotiation.cc:769] Sending connection context
0808 11:43:23.213502 (+ 288us) client_negotiation.cc:240] Negotiation successful
0808 11:43:23.214057 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":308,"threads_started":1}
I20260808 11:43:23.217267 28161 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.76.125" port: 37191 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.24.76.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.124@KRBTEST.COM" created.
Aug 08 11:43:23 dist-test-slave-tc2s krb5kdc[27901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189403, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.125@KRBTEST.COM for kudu/127.24.76.125@KRBTEST.COM
I20260808 11:43:23.231375 28166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.218113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:37191 (local address 127.0.0.1:33436)
0808 11:43:23.218308 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:23.218328 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:23.218431 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:23.219386 (+ 955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:23.219393 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:23.219738 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:23.219889 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:23.219897 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.221462 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.221467 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:23.222337 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:23.222349 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.223112 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.223118 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:23.223280 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:23.223801 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:23.223819 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:23.225948 (+ 2129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:23.229013 (+ 3065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:23.229018 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:23.229022 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:23.229360 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:23.229702 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:23.229707 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:23.229711 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:23.229851 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:23.230462 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:23.230467 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:23.230619 (+ 152us) client_negotiation.cc:769] Sending connection context
0808 11:43:23.230726 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 11:43:23.230990 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 11:43:23.231715 28171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.218352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33436 (local address 127.24.76.125:37191)
0808 11:43:23.218940 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:23.218946 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:23.218972 (+ 26us) server_negotiation.cc:407] Connection header received
0808 11:43:23.219058 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:23.219064 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:23.219235 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:23.219343 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:23.220097 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.221332 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.222461 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.222982 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.223088 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:23.226126 (+ 3038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:23.226137 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:23.226150 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:23.226182 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:23.228899 (+ 2717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:23.229483 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:23.229489 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:23.229491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:23.229606 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:23.229995 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:23.229999 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:23.230001 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:23.230343 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:23.230535 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:23.230899 (+ 364us) server_negotiation.cc:299] Negotiation successful
0808 11:43:23.231101 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":291,"threads_started":1}
I20260808 11:43:23.233886 28161 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.76.124" port: 35599 } has no permanent_uuid. Determining permanent_uuid...
I20260808 11:43:23.235169 28166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.234653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:35599 (local address 127.0.0.1:53934)
0808 11:43:23.234827 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:23.234953 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:35599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260808 11:43:23.235468 28076 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.76.124:35599: connect: Connection refused (error 111)
I20260808 11:43:23.237347 28166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.236864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:35599 (local address 127.0.0.1:53938)
0808 11:43:23.237074 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:23.237170 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:35599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260808 11:43:23.237703 28161 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.124:35599: Network error: Client connection negotiation failed: client connection to 127.24.76.124:35599: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.124.keytab.
Entry for principal kudu/127.24.76.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.124.keytab.
I20260808 11:43:23.260747 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.124:35599
--webserver_interface=127.24.76.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.124.keytab
--principal=kudu/127.24.76.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:38121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:39265,127.24.76.125:37191,127.24.76.124:35599
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:43:23.287046 28161 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.124" port: 35599 } attempt: 1
I20260808 11:43:23.288305 28166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.287825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:35599 (local address 127.0.0.1:53954)
0808 11:43:23.288039 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:23.288144 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:35599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260808 11:43:23.289414 28166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.289066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:35599 (local address 127.0.0.1:53970)
0808 11:43:23.289201 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:23.289297 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:35599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 11:43:23.289654 28161 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.124:35599: Network error: Client connection negotiation failed: client connection to 127.24.76.124:35599: connect: Connection refused (error 111)
I20260808 11:43:23.324054 28037 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.125" port: 37191 } attempt: 5
Aug 08 11:43:23 dist-test-slave-tc2s krb5kdc[27901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189402, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for kudu/127.24.76.125@KRBTEST.COM
I20260808 11:43:23.342020 28171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.325010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33444 (local address 127.24.76.125:37191)
0808 11:43:23.325226 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:23.325232 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:23.325281 (+ 49us) server_negotiation.cc:407] Connection header received
0808 11:43:23.325351 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:23.325356 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:23.325469 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:23.325585 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:23.326488 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.327639 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.331298 (+ 3659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.331989 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.332118 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:23.335509 (+ 3391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:23.335517 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:23.335522 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:23.335569 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:23.339329 (+ 3760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:23.340199 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:23.340204 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:23.340206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:23.340351 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:23.340743 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:23.340746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:23.340748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:23.341030 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:23.341140 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:23.341573 (+ 433us) server_negotiation.cc:299] Negotiation successful
0808 11:43:23.341742 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 11:43:23.342198 28039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.324845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:37191 (local address 127.0.0.1:33444)
0808 11:43:23.325068 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:23.325090 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:23.325209 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:23.325618 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:23.325621 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:23.326090 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:23.326332 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:23.326346 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.327777 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.327782 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:23.331085 (+ 3303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:23.331104 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.332160 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.332165 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:23.332378 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:23.333012 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:23.333037 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:23.335284 (+ 2247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:23.339486 (+ 4202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:23.339492 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:23.339496 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:23.340056 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:23.340458 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:23.340463 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:23.340467 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:23.340649 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:23.341151 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:23.341156 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:23.341326 (+ 170us) client_negotiation.cc:769] Sending connection context
0808 11:43:23.341510 (+ 184us) client_negotiation.cc:240] Negotiation successful
0808 11:43:23.341828 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260808 11:43:23.343338 28037 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.76.124" port: 35599 } has no permanent_uuid. Determining permanent_uuid...
I20260808 11:43:23.344369 28039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.344017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:35599 (local address 127.0.0.1:53986)
0808 11:43:23.344174 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:23.344248 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:35599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 11:43:23.345265 28039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.344954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:35599 (local address 127.0.0.1:54000)
0808 11:43:23.345093 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:23.345159 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:35599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 11:43:23.345456 28037 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.124:35599: Network error: Client connection negotiation failed: client connection to 127.24.76.124:35599: connect: Connection refused (error 111)
I20260808 11:43:23.357849 28161 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.124" port: 35599 } attempt: 2
I20260808 11:43:23.358779 28166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.358381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:35599 (local address 127.0.0.1:54008)
0808 11:43:23.358556 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:23.358637 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:35599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260808 11:43:23.359692 28166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.359353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:35599 (local address 127.0.0.1:54020)
0808 11:43:23.359509 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:23.359582 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:35599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 11:43:23.359884 28161 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.124:35599: Network error: Client connection negotiation failed: client connection to 127.24.76.124:35599: connect: Connection refused (error 111)
I20260808 11:43:23.396703 28037 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.124" port: 35599 } attempt: 1
I20260808 11:43:23.397929 28039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.397392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:35599 (local address 127.0.0.1:54028)
0808 11:43:23.397602 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:23.397695 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:35599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260808 11:43:23.398969 28039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.398640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:35599 (local address 127.0.0.1:54044)
0808 11:43:23.398785 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:23.398863 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:35599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 11:43:23.399165 28037 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.124:35599: Network error: Client connection negotiation failed: client connection to 127.24.76.124:35599: connect: Connection refused (error 111)
I20260808 11:43:23.451088 28161 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.124" port: 35599 } attempt: 3
I20260808 11:43:23.452294 28166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.451803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:35599 (local address 127.0.0.1:54048)
0808 11:43:23.452040 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:23.452133 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:35599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 11:43:23.453318 28166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.452890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:35599 (local address 127.0.0.1:54064)
0808 11:43:23.453119 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:23.453199 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:35599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
W20260808 11:43:23.453523 28161 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.124:35599: Network error: Client connection negotiation failed: client connection to 127.24.76.124:35599: connect: Connection refused (error 111)
I20260808 11:43:23.467348 28037 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.124" port: 35599 } attempt: 2
I20260808 11:43:23.468361 28039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.467906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:35599 (local address 127.0.0.1:54078)
0808 11:43:23.468113 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:23.468205 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:35599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 11:43:23.469398 28039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.469072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:35599 (local address 127.0.0.1:54090)
0808 11:43:23.469231 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:23.469296 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:35599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 11:43:23.469620 28037 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.124:35599: Network error: Client connection negotiation failed: client connection to 127.24.76.124:35599: connect: Connection refused (error 111)
I20260808 11:43:23.575830 28037 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.124" port: 35599 } attempt: 3
I20260808 11:43:23.577014 28039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.576543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:35599 (local address 127.0.0.1:54100)
0808 11:43:23.576740 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:23.576833 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:35599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 11:43:23.578130 28039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.577656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:35599 (local address 127.0.0.1:54104)
0808 11:43:23.577858 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:23.577997 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:35599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260808 11:43:23.578378 28037 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.124:35599: Network error: Client connection negotiation failed: client connection to 127.24.76.124:35599: connect: Connection refused (error 111)
I20260808 11:43:23.596719 28161 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.124" port: 35599 } attempt: 4
I20260808 11:43:23.597740 28166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.597275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:35599 (local address 127.0.0.1:54110)
0808 11:43:23.597507 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:23.597592 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:35599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 11:43:23.598879 28166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.598516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:35599 (local address 127.0.0.1:54118)
0808 11:43:23.598659 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:23.598747 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:35599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 11:43:23.599171 28161 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.124:35599: Network error: Client connection negotiation failed: client connection to 127.24.76.124:35599: connect: Connection refused (error 111)
W20260808 11:43:23.614233 28174 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 11:43:23.614540 28174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:43:23.614631 28174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:43:23.614739 28174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:43:23.621853 28174 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:43:23.621969 28174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:43:23.622062 28174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:43:23.622128 28174 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:43:23.622212 28174 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:43:23.632052 28174 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:38121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:39265,127.24.76.125:37191,127.24.76.124:35599
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.124.keytab
--principal=kudu/127.24.76.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.124:35599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.76.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:43:23.633198 28174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:43:23.634493 28174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:43:23.645283 28187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:23.645428 28192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:23.645709 28185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:43:23.646648 28174 server_base.cc:1034] running on GCE node
I20260808 11:43:23.647372 28174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:43:23.648609 28174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:43:23.649821 28174 hybrid_clock.cc:648] HybridClock initialized: now 1786189403649753 us; error 46 us; skew 500 ppm
I20260808 11:43:23.650169 28174 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:43:23 dist-test-slave-tc2s krb5kdc[27901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189403, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:43:23.653091 28174 init.cc:377] Logged in from keytab as kudu/127.24.76.124@KRBTEST.COM (short username kudu)
I20260808 11:43:23.654417 28174 webserver.cc:460] Webserver started at http://127.24.76.124:41657/ using document root <none> and password file <none>
I20260808 11:43:23.655033 28174 fs_manager.cc:359] Metadata directory not provided
I20260808 11:43:23.655115 28174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:43:23.655364 28174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:43:23.658188 28174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-2/data/instance:
uuid: "898e78341fba41619e23160907a0a1c9"
format_stamp: "Formatted at 2026-08-08 11:43:23 on dist-test-slave-tc2s"
server_key: "94d072c9fc3e75d935feae2859f2b0be"
server_key_iv: "95e18cada0dd5370f7126d93e78b5de7"
server_key_version: "encryptionkey@0"
I20260808 11:43:23.658914 28174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-2/wal/instance:
uuid: "898e78341fba41619e23160907a0a1c9"
format_stamp: "Formatted at 2026-08-08 11:43:23 on dist-test-slave-tc2s"
server_key: "94d072c9fc3e75d935feae2859f2b0be"
server_key_iv: "95e18cada0dd5370f7126d93e78b5de7"
server_key_version: "encryptionkey@0"
I20260808 11:43:23.663619 28174 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 11:43:23.666872 28208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:23.668077 28174 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:43:23.668203 28174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-2/wal
uuid: "898e78341fba41619e23160907a0a1c9"
format_stamp: "Formatted at 2026-08-08 11:43:23 on dist-test-slave-tc2s"
server_key: "94d072c9fc3e75d935feae2859f2b0be"
server_key_iv: "95e18cada0dd5370f7126d93e78b5de7"
server_key_version: "encryptionkey@0"
I20260808 11:43:23.668370 28174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:43:23.685854 28174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:43:23.689584 28174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:43:23.689797 28174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:43:23.706380 28174 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.124:35599
I20260808 11:43:23.706420 28290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.124:35599 every 8 connection(s)
I20260808 11:43:23.707634 28174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-2/data/info.pb
I20260808 11:43:23.711100 28291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:23.713315 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 28174
I20260808 11:43:23.713515 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786189042529715-24881-0/minicluster-data/master-2/wal/instance
I20260808 11:43:23.713984 24881 external_mini_cluster.cc:1428] Setting key befa58e3d6145ff31fd4840273d89a94
I20260808 11:43:23.718325 28291 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39265 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:43:23 dist-test-slave-tc2s krb5kdc[27901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.124@KRBTEST.COM
Aug 08 11:43:23 dist-test-slave-tc2s krb5kdc[27901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189403, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.124@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:43:23.734910 28297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.720991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50308 (local address 127.24.76.126:39265)
0808 11:43:23.721400 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:23.721406 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:23.721942 (+ 536us) server_negotiation.cc:407] Connection header received
0808 11:43:23.722125 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:23.722130 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:23.722206 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:23.722293 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:23.723391 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.724089 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.725506 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.725924 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.726046 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:23.729531 (+ 3485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:23.729535 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:23.729538 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:23.729579 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:23.732005 (+ 2426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:23.732652 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:23.732656 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:23.732658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:23.732781 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:23.733158 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:23.733163 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:23.733166 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:23.733425 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:23.733563 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:23.734116 (+ 553us) server_negotiation.cc:299] Negotiation successful
0808 11:43:23.734332 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":207,"threads_started":1}
I20260808 11:43:23.735582 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.721130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:39265 (local address 127.0.0.1:50308)
0808 11:43:23.721791 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:23.721831 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:23.722002 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:23.722374 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:23.722380 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:23.723003 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:23.723235 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:23.723248 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.724244 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.724248 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:23.725320 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:23.725332 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.726146 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.726154 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:23.726346 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:23.726956 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:23.726985 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:23.729340 (+ 2355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:23.732128 (+ 2788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:23.732140 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:23.732151 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:23.732541 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:23.732887 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:23.732892 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:23.732896 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:23.733052 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:23.733552 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:23.733564 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:23.733764 (+ 200us) client_negotiation.cc:769] Sending connection context
0808 11:43:23.734020 (+ 256us) client_negotiation.cc:240] Negotiation successful
0808 11:43:23.734486 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":328,"threads_started":1}
I20260808 11:43:23.735594 28296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.715731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54128 (local address 127.24.76.124:35599)
0808 11:43:23.716488 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:23.716499 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:23.716540 (+ 41us) server_negotiation.cc:407] Connection header received
0808 11:43:23.718216 (+ 1676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:23.718240 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:23.718611 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:23.718919 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:23.720776 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.722594 (+ 1818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.723804 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.724455 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.724617 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:23.729127 (+ 4510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:23.729136 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:23.729147 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:23.729177 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:23.731930 (+ 2753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:23.732590 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:23.732596 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:23.732598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:23.732740 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:23.733106 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:23.733109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:23.733111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:23.733598 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:23.733879 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:23.734233 (+ 354us) server_negotiation.cc:299] Negotiation successful
0808 11:43:23.734646 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":148,"server-negotiator.queue_time_us":554,"thread_start_us":345,"threads_started":1}
I20260808 11:43:23.737916 28291 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.76.125" port: 37191 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:43:23 dist-test-slave-tc2s krb5kdc[27901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189403, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.124@KRBTEST.COM for kudu/127.24.76.125@KRBTEST.COM
Aug 08 11:43:23 dist-test-slave-tc2s krb5kdc[27901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:43:23.750658 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.738649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:37191 (local address 127.0.0.1:33454)
0808 11:43:23.738961 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:23.738978 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:23.739070 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:23.739367 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:23.739370 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:23.739692 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:23.739829 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:23.739838 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.740896 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.740902 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:23.741813 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:23.741826 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.742636 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.742642 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:23.742833 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:23.743391 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:23.743408 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:23.745190 (+ 1782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:23.748275 (+ 3085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:23.748280 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:23.748284 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:23.748756 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:23.749104 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:23.749109 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:23.749113 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:23.749282 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:23.749730 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:23.749735 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:23.749926 (+ 191us) client_negotiation.cc:769] Sending connection context
0808 11:43:23.750098 (+ 172us) client_negotiation.cc:240] Negotiation successful
0808 11:43:23.750328 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211}
I20260808 11:43:23.750717 28171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.738648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33454 (local address 127.24.76.125:37191)
0808 11:43:23.738834 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:23.738839 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:23.739050 (+ 211us) server_negotiation.cc:407] Connection header received
0808 11:43:23.739176 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:23.739181 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:23.739259 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:23.739363 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:23.739994 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.740766 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.741975 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.742499 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.742589 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:23.745367 (+ 2778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:23.745372 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:23.745375 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:23.745407 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:23.748155 (+ 2748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:23.748850 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:23.748853 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:23.748855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:23.749004 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:23.749372 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:23.749375 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:23.749377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:23.749607 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:23.749721 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:23.750185 (+ 464us) server_negotiation.cc:299] Negotiation successful
0808 11:43:23.750407 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 11:43:23.751910 28291 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.76.124" port: 35599 } has no permanent_uuid. Determining permanent_uuid...
I20260808 11:43:23.752197 28297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.738296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50324 (local address 127.24.76.126:39265)
0808 11:43:23.738465 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:23.738470 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:23.738703 (+ 233us) server_negotiation.cc:407] Connection header received
0808 11:43:23.738768 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:23.738773 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:23.738850 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:23.738941 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:23.740249 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.740964 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.742266 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.742728 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.742847 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:23.746671 (+ 3824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:23.746676 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:23.746679 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:23.746709 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:23.749431 (+ 2722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:23.750155 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:23.750160 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:23.750163 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:23.750311 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:23.750761 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:23.750766 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:23.750769 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:23.751088 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:23.751563 (+ 475us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:23.751714 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 11:43:23.751923 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 11:43:23.755611 28037 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.124" port: 35599 } attempt: 4
Aug 08 11:43:23 dist-test-slave-tc2s krb5kdc[27901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189403, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.124@KRBTEST.COM for kudu/127.24.76.124@KRBTEST.COM
Aug 08 11:43:23 dist-test-slave-tc2s krb5kdc[27901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.125@KRBTEST.COM
Aug 08 11:43:23 dist-test-slave-tc2s krb5kdc[27901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189402, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for kudu/127.24.76.124@KRBTEST.COM
I20260808 11:43:23.766546 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.752702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:35599 (local address 127.0.0.1:54134)
0808 11:43:23.752893 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:23.752907 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:23.752993 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:23.753276 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:23.753280 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:23.753678 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:23.753846 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:23.753859 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.754770 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.754775 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:23.755651 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:23.755666 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.756790 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.756794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:23.756946 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:23.757829 (+ 883us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:23.757851 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:23.760257 (+ 2406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:23.763982 (+ 3725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:23.763986 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:23.763990 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:23.764537 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:23.764902 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:23.764907 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:23.764911 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:23.765038 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:23.765528 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:23.765533 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:23.765673 (+ 140us) client_negotiation.cc:769] Sending connection context
0808 11:43:23.765821 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 11:43:23.766139 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121}
I20260808 11:43:23.767081 28296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.752678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54134 (local address 127.24.76.124:35599)
0808 11:43:23.752842 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:23.752848 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:23.752967 (+ 119us) server_negotiation.cc:407] Connection header received
0808 11:43:23.753093 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:23.753099 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:23.753157 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:23.753240 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:23.754001 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.754656 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.755798 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.756501 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.756629 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:23.760519 (+ 3890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:23.760527 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:23.760532 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:23.760568 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:23.763819 (+ 3251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:23.764654 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:23.764662 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:23.764666 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:23.764799 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:23.765136 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:23.765140 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:23.765144 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:23.765409 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:23.765541 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:23.765940 (+ 399us) server_negotiation.cc:299] Negotiation successful
0808 11:43:23.766791 (+ 851us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":482,"server-negotiator.queue_time_us":49}
I20260808 11:43:23.770402 28171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.753750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33460 (local address 127.24.76.125:37191)
0808 11:43:23.753932 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:23.753937 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:23.753961 (+ 24us) server_negotiation.cc:407] Connection header received
0808 11:43:23.754116 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:23.754121 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:23.754177 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:23.754248 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:23.755359 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.756622 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.758286 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.758914 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.759046 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:23.764624 (+ 5578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:23.764630 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:23.764635 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:23.764672 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:23.767953 (+ 3281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:23.768548 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:23.768553 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:23.768557 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:23.768703 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:23.769060 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:23.769066 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:23.769069 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:23.769321 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:23.769573 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:23.769861 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 11:43:23.770102 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 11:43:23.772334 28291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 898e78341fba41619e23160907a0a1c9: Bootstrap starting.
I20260808 11:43:23.772681 28039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.756088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:35599 (local address 127.0.0.1:54140)
0808 11:43:23.756248 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:23.756265 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:23.756358 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:23.757148 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:23.757152 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:23.757583 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:23.757755 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:23.757767 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.759110 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.759115 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:23.760208 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:23.760221 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.760944 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.760950 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:23.761139 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:23.761626 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:23.761642 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:23.763942 (+ 2300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:23.769965 (+ 6023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:23.769971 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:23.769975 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:23.770442 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:23.770895 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:23.770900 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:23.770903 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:23.771068 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:23.771688 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:23.771693 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:23.771890 (+ 197us) client_negotiation.cc:769] Sending connection context
0808 11:43:23.772043 (+ 153us) client_negotiation.cc:240] Negotiation successful
0808 11:43:23.772311 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 11:43:23.773075 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.756146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54140 (local address 127.24.76.124:35599)
0808 11:43:23.756830 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:23.756835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:23.756860 (+ 25us) server_negotiation.cc:407] Connection header received
0808 11:43:23.756911 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:23.756916 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:23.756982 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:23.757097 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:23.757952 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.758947 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.760374 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.760804 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.760921 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:23.764158 (+ 3237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:23.764164 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:23.764167 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:23.764195 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:23.767785 (+ 3590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:23.770594 (+ 2809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:23.770598 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:23.770601 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:23.770760 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:23.771199 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:23.771203 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:23.771206 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:23.771553 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:23.771689 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:23.772126 (+ 437us) server_negotiation.cc:299] Negotiation successful
0808 11:43:23.772307 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":355,"threads_started":1}
I20260808 11:43:23.775416 24881 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:43:23.776225 28291 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 898e78341fba41619e23160907a0a1c9: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:23.777245 28291 log.cc:824] T 00000000000000000000000000000000 P 898e78341fba41619e23160907a0a1c9: Log is configured to *not* fsync() on all Append() calls
I20260808 11:43:23.778812 28037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be0704aff1c74ec8b62f7040b26dcace: Bootstrap starting.
I20260808 11:43:23.780326 28291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 898e78341fba41619e23160907a0a1c9: No bootstrap required, opened a new log
I20260808 11:43:23.782137 28037 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be0704aff1c74ec8b62f7040b26dcace: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:23.778919 27992 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "be0704aff1c74ec8b62f7040b26dcace"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50324
I20260808 11:43:23.783193 28037 log.cc:824] T 00000000000000000000000000000000 P be0704aff1c74ec8b62f7040b26dcace: Log is configured to *not* fsync() on all Append() calls
I20260808 11:43:23.786255 28037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be0704aff1c74ec8b62f7040b26dcace: No bootstrap required, opened a new log
I20260808 11:43:23.788821 28291 raft_consensus.cc:348] T 00000000000000000000000000000000 P 898e78341fba41619e23160907a0a1c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0704aff1c74ec8b62f7040b26dcace" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39265 } } peers { permanent_uuid: "10153b1ecada481abfb14380b258d382" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 37191 } } peers { permanent_uuid: "898e78341fba41619e23160907a0a1c9" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 35599 } }
I20260808 11:43:23.789156 28291 raft_consensus.cc:729] T 00000000000000000000000000000000 P 898e78341fba41619e23160907a0a1c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 898e78341fba41619e23160907a0a1c9, State: Initialized, Role: FOLLOWER
I20260808 11:43:23.789638 28291 consensus_queue.cc:260] T 00000000000000000000000000000000 P 898e78341fba41619e23160907a0a1c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0704aff1c74ec8b62f7040b26dcace" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39265 } } peers { permanent_uuid: "10153b1ecada481abfb14380b258d382" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 37191 } } peers { permanent_uuid: "898e78341fba41619e23160907a0a1c9" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 35599 } }
I20260808 11:43:23.790592 28309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 898e78341fba41619e23160907a0a1c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0704aff1c74ec8b62f7040b26dcace" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39265 } } peers { permanent_uuid: "10153b1ecada481abfb14380b258d382" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 37191 } } peers { permanent_uuid: "898e78341fba41619e23160907a0a1c9" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 35599 } } }
I20260808 11:43:23.791054 28309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 898e78341fba41619e23160907a0a1c9 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:43:23.791714 28291 sys_catalog.cc:564] T 00000000000000000000000000000000 P 898e78341fba41619e23160907a0a1c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:43:23.793237 28037 raft_consensus.cc:348] T 00000000000000000000000000000000 P be0704aff1c74ec8b62f7040b26dcace [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0704aff1c74ec8b62f7040b26dcace" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39265 } } peers { permanent_uuid: "10153b1ecada481abfb14380b258d382" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 37191 } } peers { permanent_uuid: "898e78341fba41619e23160907a0a1c9" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 35599 } }
I20260808 11:43:23.793591 28037 raft_consensus.cc:729] T 00000000000000000000000000000000 P be0704aff1c74ec8b62f7040b26dcace [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be0704aff1c74ec8b62f7040b26dcace, State: Initialized, Role: FOLLOWER
I20260808 11:43:23.794041 28037 consensus_queue.cc:260] T 00000000000000000000000000000000 P be0704aff1c74ec8b62f7040b26dcace [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0704aff1c74ec8b62f7040b26dcace" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39265 } } peers { permanent_uuid: "10153b1ecada481abfb14380b258d382" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 37191 } } peers { permanent_uuid: "898e78341fba41619e23160907a0a1c9" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 35599 } }
I20260808 11:43:23.794905 28316 sys_catalog.cc:455] T 00000000000000000000000000000000 P be0704aff1c74ec8b62f7040b26dcace [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0704aff1c74ec8b62f7040b26dcace" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39265 } } peers { permanent_uuid: "10153b1ecada481abfb14380b258d382" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 37191 } } peers { permanent_uuid: "898e78341fba41619e23160907a0a1c9" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 35599 } } }
I20260808 11:43:23.795284 28316 sys_catalog.cc:458] T 00000000000000000000000000000000 P be0704aff1c74ec8b62f7040b26dcace [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:43:23.795878 28037 sys_catalog.cc:564] T 00000000000000000000000000000000 P be0704aff1c74ec8b62f7040b26dcace [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:43:23.795979 27992 raft_consensus.cc:482] T 00000000000000000000000000000000 P be0704aff1c74ec8b62f7040b26dcace [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 11:43:23.796193 27992 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be0704aff1c74ec8b62f7040b26dcace [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:23.799890 27992 raft_consensus.cc:504] T 00000000000000000000000000000000 P be0704aff1c74ec8b62f7040b26dcace [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0704aff1c74ec8b62f7040b26dcace" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39265 } } peers { permanent_uuid: "10153b1ecada481abfb14380b258d382" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 37191 } } peers { permanent_uuid: "898e78341fba41619e23160907a0a1c9" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 35599 } }
I20260808 11:43:23.803467 27992 leader_election.cc:288] T 00000000000000000000000000000000 P be0704aff1c74ec8b62f7040b26dcace [CANDIDATE]: Term 1 election: Requested vote from peers 10153b1ecada481abfb14380b258d382 (127.24.76.125:37191), 898e78341fba41619e23160907a0a1c9 (127.24.76.124:35599)
I20260808 11:43:23.803732 28251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "be0704aff1c74ec8b62f7040b26dcace" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "898e78341fba41619e23160907a0a1c9"
I20260808 11:43:23.804200 28251 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 898e78341fba41619e23160907a0a1c9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:23.807672 28251 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 898e78341fba41619e23160907a0a1c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be0704aff1c74ec8b62f7040b26dcace in term 1.
I20260808 11:43:23.808301 27952 leader_election.cc:302] T 00000000000000000000000000000000 P be0704aff1c74ec8b62f7040b26dcace [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 898e78341fba41619e23160907a0a1c9, be0704aff1c74ec8b62f7040b26dcace; no voters:
I20260808 11:43:23.808763 28316 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be0704aff1c74ec8b62f7040b26dcace [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:43:23.805085 28174 master_runner.cc:186] Error getting master registration for 127.24.76.126:39265: OK, instance_id { permanent_uuid: "be0704aff1c74ec8b62f7040b26dcace" instance_seqno: 1786189402616735 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 11:43:23.816176 28174 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 11:43:23.820520 28316 raft_consensus.cc:686] T 00000000000000000000000000000000 P be0704aff1c74ec8b62f7040b26dcace [term 1 LEADER]: Becoming Leader. State: Replica: be0704aff1c74ec8b62f7040b26dcace, State: Running, Role: LEADER
I20260808 11:43:23.821171 28316 consensus_queue.cc:237] T 00000000000000000000000000000000 P be0704aff1c74ec8b62f7040b26dcace [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0704aff1c74ec8b62f7040b26dcace" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39265 } } peers { permanent_uuid: "10153b1ecada481abfb14380b258d382" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 37191 } } peers { permanent_uuid: "898e78341fba41619e23160907a0a1c9" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 35599 } }
I20260808 11:43:23.825736 28345 sys_catalog.cc:455] T 00000000000000000000000000000000 P be0704aff1c74ec8b62f7040b26dcace [sys.catalog]: SysCatalogTable state changed. Reason: New leader be0704aff1c74ec8b62f7040b26dcace. Latest consensus state: current_term: 1 leader_uuid: "be0704aff1c74ec8b62f7040b26dcace" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0704aff1c74ec8b62f7040b26dcace" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39265 } } peers { permanent_uuid: "10153b1ecada481abfb14380b258d382" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 37191 } } peers { permanent_uuid: "898e78341fba41619e23160907a0a1c9" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 35599 } } }
I20260808 11:43:23.826707 28345 sys_catalog.cc:458] T 00000000000000000000000000000000 P be0704aff1c74ec8b62f7040b26dcace [sys.catalog]: This master's current role is: LEADER
I20260808 11:43:23.830861 28351 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260808 11:43:23.831182 28338 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 898e78341fba41619e23160907a0a1c9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 11:43:23.831305 28338 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 11:43:23.841687 28351 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260808 11:43:23.857550 27958 tablet.cc:2369] T 00000000000000000000000000000000 P be0704aff1c74ec8b62f7040b26dcace: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:43:23.878192 28251 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 898e78341fba41619e23160907a0a1c9 [term 1 FOLLOWER]: Refusing update from remote peer be0704aff1c74ec8b62f7040b26dcace: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:43:23.879323 28316 consensus_queue.cc:1035] T 00000000000000000000000000000000 P be0704aff1c74ec8b62f7040b26dcace [LEADER]: Connected to new peer: Peer: permanent_uuid: "898e78341fba41619e23160907a0a1c9" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:43:23.880733 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.835152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54142 (local address 127.24.76.124:35599)
0808 11:43:23.835336 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:23.835343 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:23.842450 (+ 7107us) server_negotiation.cc:407] Connection header received
0808 11:43:23.842756 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:23.842764 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:23.847844 (+ 5080us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:23.847964 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:23.850131 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.851151 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.854071 (+ 2920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.854707 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.854805 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:23.873319 (+ 18514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:23.873326 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:23.873331 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:23.873376 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:23.876810 (+ 3434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:23.877734 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:23.877740 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:23.877743 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:23.877928 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:23.878392 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:23.878397 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:23.878400 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:23.878808 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:23.879049 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:23.880149 (+ 1100us) server_negotiation.cc:299] Negotiation successful
0808 11:43:23.880389 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 11:43:23.866415 27924 master_runner.cc:186] Error getting master registration for 127.24.76.125:37191: OK, instance_id { permanent_uuid: "10153b1ecada481abfb14380b258d382" instance_seqno: 1786189403172616 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 11:43:23.890384 27924 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 11:43:23.897483 28297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.835237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50326 (local address 127.24.76.126:39265)
0808 11:43:23.835397 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:23.835402 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:23.843949 (+ 8547us) server_negotiation.cc:407] Connection header received
0808 11:43:23.844729 (+ 780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:23.844736 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:23.844846 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:23.844929 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:23.848843 (+ 3914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.850081 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.855085 (+ 5004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.855765 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.855878 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:23.875209 (+ 19331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:23.875217 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:23.875222 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:23.875268 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:23.882839 (+ 7571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:23.885580 (+ 2741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:23.885586 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:23.885685 (+ 99us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:23.886277 (+ 592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:23.887213 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:23.887219 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:23.887223 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:23.888707 (+ 1484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:23.889645 (+ 938us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:23.890292 (+ 647us) server_negotiation.cc:299] Negotiation successful
0808 11:43:23.895845 (+ 5553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 11:43:23.898965 28161 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.124" port: 35599 } attempt: 5
I20260808 11:43:23.905318 28309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 898e78341fba41619e23160907a0a1c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be0704aff1c74ec8b62f7040b26dcace. Latest consensus state: current_term: 1 leader_uuid: "be0704aff1c74ec8b62f7040b26dcace" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0704aff1c74ec8b62f7040b26dcace" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39265 } } peers { permanent_uuid: "10153b1ecada481abfb14380b258d382" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 37191 } } peers { permanent_uuid: "898e78341fba41619e23160907a0a1c9" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 35599 } } }
I20260808 11:43:23.905622 28309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 898e78341fba41619e23160907a0a1c9 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:43:23.908407 28171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.836121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33464 (local address 127.24.76.125:37191)
0808 11:43:23.836298 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:23.836304 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:23.853428 (+ 17124us) server_negotiation.cc:407] Connection header received
0808 11:43:23.854094 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:23.854099 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:23.854185 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:23.854258 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:23.872797 (+ 18539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.873929 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.880066 (+ 6137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.880721 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.880851 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:23.890873 (+ 10022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:23.890881 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:23.890886 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:23.890936 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:23.895017 (+ 4081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:23.903973 (+ 8956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:23.903980 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:23.903984 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:23.904253 (+ 269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:23.905081 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:23.905109 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:23.905113 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:23.905697 (+ 584us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:23.906073 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:23.907101 (+ 1028us) server_negotiation.cc:299] Negotiation successful
0808 11:43:23.907604 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
Aug 08 11:43:23 dist-test-slave-tc2s krb5kdc[27901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189403, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.125@KRBTEST.COM for kudu/127.24.76.124@KRBTEST.COM
I20260808 11:43:23.916891 28363 mvcc.cc:205] Tried to move back new op lower bound from 7316231798252888064 to 7316231798057086976. Current Snapshot: MvccSnapshot[applied={T|T < 7316231798252888064}]
I20260808 11:43:23.920126 28166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.899522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:35599 (local address 127.0.0.1:54148)
0808 11:43:23.899890 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:23.899909 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:23.900011 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:23.900986 (+ 975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:23.900991 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:23.901466 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:23.901665 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:23.901677 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.903650 (+ 1973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.903656 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:23.904958 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:23.904975 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.906321 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.906327 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:23.906577 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:23.907208 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:23.907232 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:23.910413 (+ 3181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:23.916419 (+ 6006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:23.916425 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:23.916430 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:23.916953 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:23.917408 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:23.917413 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:23.917416 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:23.917578 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:23.918378 (+ 800us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:23.918383 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:23.918543 (+ 160us) client_negotiation.cc:769] Sending connection context
0808 11:43:23.919425 (+ 882us) client_negotiation.cc:240] Negotiation successful
0808 11:43:23.919735 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231}
I20260808 11:43:23.921435 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:23.900240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54148 (local address 127.24.76.124:35599)
0808 11:43:23.900475 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:23.900482 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:23.900500 (+ 18us) server_negotiation.cc:407] Connection header received
0808 11:43:23.900558 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:23.900563 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:23.900635 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:23.900727 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:23.902263 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.903493 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.905139 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:23.905830 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:23.906081 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:23.912384 (+ 6303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:23.912391 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:23.912396 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:23.912433 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:23.916255 (+ 3822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:23.917097 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:23.917107 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:23.917111 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:23.917275 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:23.917713 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:23.917717 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:23.917721 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:23.918128 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:23.918302 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:23.920575 (+ 2273us) server_negotiation.cc:299] Negotiation successful
0808 11:43:23.921035 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260808 11:43:23.925521 28309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 898e78341fba41619e23160907a0a1c9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "be0704aff1c74ec8b62f7040b26dcace" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0704aff1c74ec8b62f7040b26dcace" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39265 } } peers { permanent_uuid: "10153b1ecada481abfb14380b258d382" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 37191 } } peers { permanent_uuid: "898e78341fba41619e23160907a0a1c9" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 35599 } } }
I20260808 11:43:23.926009 28309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 898e78341fba41619e23160907a0a1c9 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:43:23.927500 28351 catalog_manager.cc:1347] Generated new cluster ID: 892f7793c0e24d0bb7b0cf1cf714d299
I20260808 11:43:23.927620 28351 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:43:23.929183 28161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10153b1ecada481abfb14380b258d382: Bootstrap starting.
I20260808 11:43:23.932621 28316 sys_catalog.cc:455] T 00000000000000000000000000000000 P be0704aff1c74ec8b62f7040b26dcace [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "be0704aff1c74ec8b62f7040b26dcace" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0704aff1c74ec8b62f7040b26dcace" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39265 } } peers { permanent_uuid: "10153b1ecada481abfb14380b258d382" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 37191 } } peers { permanent_uuid: "898e78341fba41619e23160907a0a1c9" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 35599 } } }
I20260808 11:43:23.932930 28316 sys_catalog.cc:458] T 00000000000000000000000000000000 P be0704aff1c74ec8b62f7040b26dcace [sys.catalog]: This master's current role is: LEADER
I20260808 11:43:23.939661 28362 mvcc.cc:205] Tried to move back new op lower bound from 7316231798252888064 to 7316231798057086976. Current Snapshot: MvccSnapshot[applied={T|T < 7316231798252888064 or (T in {7316231798252888064})}]
I20260808 11:43:23.943116 28161 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10153b1ecada481abfb14380b258d382: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:23.946092 28161 log.cc:824] T 00000000000000000000000000000000 P 10153b1ecada481abfb14380b258d382: Log is configured to *not* fsync() on all Append() calls
I20260808 11:43:23.947597 28351 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:43:23.949366 28351 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:43:23.961277 28161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10153b1ecada481abfb14380b258d382: No bootstrap required, opened a new log
I20260808 11:43:23.973692 28161 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10153b1ecada481abfb14380b258d382 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0704aff1c74ec8b62f7040b26dcace" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39265 } } peers { permanent_uuid: "10153b1ecada481abfb14380b258d382" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 37191 } } peers { permanent_uuid: "898e78341fba41619e23160907a0a1c9" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 35599 } }
I20260808 11:43:23.975534 28351 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be0704aff1c74ec8b62f7040b26dcace: Generated new TSK 0
I20260808 11:43:23.974447 28161 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10153b1ecada481abfb14380b258d382 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10153b1ecada481abfb14380b258d382, State: Initialized, Role: FOLLOWER
I20260808 11:43:23.977684 28161 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10153b1ecada481abfb14380b258d382 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0704aff1c74ec8b62f7040b26dcace" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39265 } } peers { permanent_uuid: "10153b1ecada481abfb14380b258d382" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 37191 } } peers { permanent_uuid: "898e78341fba41619e23160907a0a1c9" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 35599 } }
I20260808 11:43:23.981870 28351 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:43:23.991964 28373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10153b1ecada481abfb14380b258d382 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0704aff1c74ec8b62f7040b26dcace" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39265 } } peers { permanent_uuid: "10153b1ecada481abfb14380b258d382" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 37191 } } peers { permanent_uuid: "898e78341fba41619e23160907a0a1c9" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 35599 } } }
I20260808 11:43:23.992516 28373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10153b1ecada481abfb14380b258d382 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:43:23.993429 28161 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10153b1ecada481abfb14380b258d382 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:43:23.995201 28117 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10153b1ecada481abfb14380b258d382 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:23.998090 28118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "be0704aff1c74ec8b62f7040b26dcace" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "10153b1ecada481abfb14380b258d382"
I20260808 11:43:24.000823 28117 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 10153b1ecada481abfb14380b258d382 [term 1 FOLLOWER]: Refusing update from remote peer be0704aff1c74ec8b62f7040b26dcace: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:43:24.004568 28118 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 10153b1ecada481abfb14380b258d382 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be0704aff1c74ec8b62f7040b26dcace in term 1.
I20260808 11:43:24.007391 28345 consensus_queue.cc:1035] T 00000000000000000000000000000000 P be0704aff1c74ec8b62f7040b26dcace [LEADER]: Connected to new peer: Peer: permanent_uuid: "10153b1ecada481abfb14380b258d382" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 37191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:43:24.062161 28389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10153b1ecada481abfb14380b258d382 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "be0704aff1c74ec8b62f7040b26dcace" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0704aff1c74ec8b62f7040b26dcace" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39265 } } peers { permanent_uuid: "10153b1ecada481abfb14380b258d382" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 37191 } } peers { permanent_uuid: "898e78341fba41619e23160907a0a1c9" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 35599 } } }
I20260808 11:43:24.062666 28389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10153b1ecada481abfb14380b258d382 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:43:24.062849 28373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10153b1ecada481abfb14380b258d382 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be0704aff1c74ec8b62f7040b26dcace. Latest consensus state: current_term: 1 leader_uuid: "be0704aff1c74ec8b62f7040b26dcace" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0704aff1c74ec8b62f7040b26dcace" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39265 } } peers { permanent_uuid: "10153b1ecada481abfb14380b258d382" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 37191 } } peers { permanent_uuid: "898e78341fba41619e23160907a0a1c9" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 35599 } } }
I20260808 11:43:24.063124 28373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10153b1ecada481abfb14380b258d382 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:43:24.080199 28402 catalog_manager.cc:1259] Loaded cluster ID: 892f7793c0e24d0bb7b0cf1cf714d299
I20260808 11:43:24.080327 28402 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 10153b1ecada481abfb14380b258d382: loading cluster ID for follower catalog manager: success
I20260808 11:43:24.084344 28402 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 10153b1ecada481abfb14380b258d382: acquiring CA information for follower catalog manager: success
I20260808 11:43:24.087033 28402 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 10153b1ecada481abfb14380b258d382: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 11:43:24.114610 24881 mini_kdc.cc:348] Time spent kdestroy: real 0.112s user 0.001s sys 0.029s
W20260808 11:43:24.171334 28171 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:43:24.147741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33476 (local address 127.24.76.125:37191)
0808 11:43:24.147967 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:24.147974 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:24.167655 (+ 19681us) server_negotiation.cc:407] Connection header received
0808 11:43:24.169654 (+ 1999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:24.169661 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:24.169767 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:24.169854 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:24.171075 (+ 1221us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33476: BlockingRecv error: recv got EOF from 127.0.0.1:33476 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260808 11:43:24.173873 28304 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:43:24.150169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54156 (local address 127.24.76.124:35599)
0808 11:43:24.150368 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:24.150374 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:24.169159 (+ 18785us) server_negotiation.cc:407] Connection header received
0808 11:43:24.172219 (+ 3060us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:24.172225 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:24.172343 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:24.172421 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:24.173619 (+ 1198us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54156: BlockingRecv error: recv got EOF from 127.0.0.1:54156 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260808 11:43:24.176743 28297 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:43:24.164175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50340 (local address 127.24.76.126:39265)
0808 11:43:24.164386 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:24.164392 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:24.174598 (+ 10206us) server_negotiation.cc:407] Connection header received
0808 11:43:24.175469 (+ 871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:24.175475 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:24.175559 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:24.175629 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:24.176506 (+ 877us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50340: BlockingRecv error: recv got EOF from 127.0.0.1:50340 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 11:43:24.229324 28171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:24.205736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33484 (local address 127.24.76.125:37191)
0808 11:43:24.205939 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:24.205945 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:24.213693 (+ 7748us) server_negotiation.cc:407] Connection header received
0808 11:43:24.214301 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:24.214307 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:24.214488 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:24.214614 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:24.218178 (+ 3564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:24.219365 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:24.225993 (+ 6628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:24.226633 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:24.226739 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:24.227618 (+ 879us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:24.228326 (+ 708us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:24.228438 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:24.228821 (+ 383us) server_negotiation.cc:299] Negotiation successful
0808 11:43:24.228937 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260808 11:43:24.282692 28304 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:43:24.279857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54164 (local address 127.24.76.124:35599)
0808 11:43:24.280066 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:24.280073 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:24.280599 (+ 526us) server_negotiation.cc:407] Connection header received
0808 11:43:24.280986 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:24.280992 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:24.281130 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:24.281222 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:24.282437 (+ 1215us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54164: BlockingRecv error: recv got EOF from 127.0.0.1:54164 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 11:43:24.332167 28171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:24.316806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33494 (local address 127.24.76.125:37191)
0808 11:43:24.317036 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:24.317044 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:24.324536 (+ 7492us) server_negotiation.cc:407] Connection header received
0808 11:43:24.325145 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:24.325152 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:24.325279 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:24.325356 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:24.326291 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:24.327491 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:24.329712 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:24.330477 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:24.330586 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:24.331247 (+ 661us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:24.331425 (+ 178us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:24.331492 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:24.331749 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 11:43:24.331840 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260808 11:43:24.379733 28304 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:43:24.366340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54178 (local address 127.24.76.124:35599)
0808 11:43:24.366532 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:24.366539 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:24.377425 (+ 10886us) server_negotiation.cc:407] Connection header received
0808 11:43:24.378023 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:24.378030 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:24.378137 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:24.378234 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:24.379456 (+ 1222us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54178: BlockingRecv error: recv got EOF from 127.0.0.1:54178 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 11:43:24.431572 28171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:24.416675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33504 (local address 127.24.76.125:37191)
0808 11:43:24.416866 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:24.416873 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:24.424308 (+ 7435us) server_negotiation.cc:407] Connection header received
0808 11:43:24.424870 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:24.424876 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:24.424984 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:24.425056 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:24.425930 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:24.427129 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:24.429182 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:24.429859 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:24.429993 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:24.430663 (+ 670us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:24.430807 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:24.430865 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:24.431119 (+ 254us) server_negotiation.cc:299] Negotiation successful
0808 11:43:24.431221 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260808 11:43:24.468971 28304 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:43:24.465217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54188 (local address 127.24.76.124:35599)
0808 11:43:24.465396 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:24.465402 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:24.467318 (+ 1916us) server_negotiation.cc:407] Connection header received
0808 11:43:24.467556 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:24.467562 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:24.467661 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:24.467780 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:24.468710 (+ 930us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54188: BlockingRecv error: recv got EOF from 127.0.0.1:54188 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 11:43:24.481057 28171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:24.476062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33506 (local address 127.24.76.125:37191)
0808 11:43:24.476244 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:24.476250 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:24.476597 (+ 347us) server_negotiation.cc:407] Connection header received
0808 11:43:24.476766 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:24.476770 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:24.476837 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:24.476924 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:24.477319 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:24.478160 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:24.479460 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:24.479898 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:24.480006 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:24.480400 (+ 394us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:24.480515 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:24.480586 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:24.480733 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 11:43:24.480813 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260808 11:43:24.492605 28304 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:43:24.490673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54202 (local address 127.24.76.124:35599)
0808 11:43:24.490880 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:24.490885 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:24.491224 (+ 339us) server_negotiation.cc:407] Connection header received
0808 11:43:24.491388 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:24.491392 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:24.491465 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:24.491546 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:24.492388 (+ 842us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54202: BlockingRecv error: recv got EOF from 127.0.0.1:54202 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
I20260808 11:43:24.510066 28171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:24.504386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33520 (local address 127.24.76.125:37191)
0808 11:43:24.504594 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:24.504600 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:24.504883 (+ 283us) server_negotiation.cc:407] Connection header received
0808 11:43:24.505077 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:24.505081 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:24.505144 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:24.505222 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:24.505707 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:24.506493 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:24.508333 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:24.508754 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:24.508841 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:24.509317 (+ 476us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:24.509427 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:24.509491 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:24.509675 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 11:43:24.509737 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260808 11:43:24.518321 28304 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:43:24.516511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54208 (local address 127.24.76.124:35599)
0808 11:43:24.516726 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:24.516732 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:24.517049 (+ 317us) server_negotiation.cc:407] Connection header received
0808 11:43:24.517185 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:24.517191 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:24.517283 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:24.517404 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:24.518108 (+ 704us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54208: BlockingRecv error: recv got EOF from 127.0.0.1:54208 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 11:43:24.543407 28171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:24.537586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33522 (local address 127.24.76.125:37191)
0808 11:43:24.537817 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:24.537826 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:24.538224 (+ 398us) server_negotiation.cc:407] Connection header received
0808 11:43:24.538398 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:24.538403 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:24.538498 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:24.538640 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:24.539018 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:24.540199 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:24.541448 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:24.542156 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:24.542283 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:24.542706 (+ 423us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:24.542842 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:24.542934 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:24.543033 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 11:43:24.543115 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 11:43:24.544219 28345 sys_catalog.cc:455] T 00000000000000000000000000000000 P be0704aff1c74ec8b62f7040b26dcace [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "be0704aff1c74ec8b62f7040b26dcace" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0704aff1c74ec8b62f7040b26dcace" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 39265 } } peers { permanent_uuid: "10153b1ecada481abfb14380b258d382" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 37191 } } peers { permanent_uuid: "898e78341fba41619e23160907a0a1c9" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 35599 } } }
I20260808 11:43:24.544545 28345 sys_catalog.cc:458] T 00000000000000000000000000000000 P be0704aff1c74ec8b62f7040b26dcace [sys.catalog]: This master's current role is: LEADER
W20260808 11:43:24.552280 28304 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:43:24.550587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54220 (local address 127.24.76.124:35599)
0808 11:43:24.550767 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:24.550773 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:24.550985 (+ 212us) server_negotiation.cc:407] Connection header received
0808 11:43:24.551144 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:24.551148 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:24.551221 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:24.551298 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:24.552073 (+ 775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54220: BlockingRecv error: recv got EOF from 127.0.0.1:54220 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 11:43:24.592970 28171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:24.588260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33530 (local address 127.24.76.125:37191)
0808 11:43:24.588460 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:24.588465 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:24.588812 (+ 347us) server_negotiation.cc:407] Connection header received
0808 11:43:24.588996 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:24.589001 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:24.589067 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:24.589136 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:24.589461 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:24.590191 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:24.591481 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:24.591933 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:24.592030 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:24.592395 (+ 365us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:24.592494 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:24.592556 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:24.592668 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 11:43:24.592729 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260808 11:43:24.600754 28304 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:43:24.598776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54222 (local address 127.24.76.124:35599)
0808 11:43:24.598943 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:24.598949 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:24.599503 (+ 554us) server_negotiation.cc:407] Connection header received
0808 11:43:24.599660 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:24.599664 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:24.599743 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:24.599845 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:24.600537 (+ 692us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54222: BlockingRecv error: recv got EOF from 127.0.0.1:54222 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 11:43:24.672941 28171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:24.668302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33536 (local address 127.24.76.125:37191)
0808 11:43:24.668529 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:24.668535 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:24.668791 (+ 256us) server_negotiation.cc:407] Connection header received
0808 11:43:24.668944 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:24.668950 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:24.669020 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:24.669097 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:24.669428 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:24.670207 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:24.671410 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:24.671800 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:24.671883 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:24.672346 (+ 463us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:24.672462 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:24.672526 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:24.672622 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 11:43:24.672681 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260808 11:43:24.679760 28304 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:43:24.678015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54226 (local address 127.24.76.124:35599)
0808 11:43:24.678189 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:24.678196 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:24.678548 (+ 352us) server_negotiation.cc:407] Connection header received
0808 11:43:24.678707 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:24.678711 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:24.678781 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:24.678861 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:24.679604 (+ 743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54226: BlockingRecv error: recv got EOF from 127.0.0.1:54226 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 11:43:24.816676 28171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:24.811360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33552 (local address 127.24.76.125:37191)
0808 11:43:24.811619 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:24.811626 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:24.811921 (+ 295us) server_negotiation.cc:407] Connection header received
0808 11:43:24.812137 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:24.812141 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:24.812213 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:24.812298 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:24.812761 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:24.813469 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:24.815029 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:24.815460 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:24.815578 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:24.815987 (+ 409us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:24.816097 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:24.816171 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:24.816323 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 11:43:24.816406 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
W20260808 11:43:24.823933 28304 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:43:24.822164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54234 (local address 127.24.76.124:35599)
0808 11:43:24.822394 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:24.822399 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:24.822652 (+ 253us) server_negotiation.cc:407] Connection header received
0808 11:43:24.822800 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:24.822804 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:24.822876 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:24.822960 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:24.823671 (+ 711us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54234: BlockingRecv error: recv got EOF from 127.0.0.1:54234 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 11:43:24.833312 28338 catalog_manager.cc:1259] Loaded cluster ID: 892f7793c0e24d0bb7b0cf1cf714d299
I20260808 11:43:24.833417 28338 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 898e78341fba41619e23160907a0a1c9: loading cluster ID for follower catalog manager: success
I20260808 11:43:24.836352 28338 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 898e78341fba41619e23160907a0a1c9: acquiring CA information for follower catalog manager: success
I20260808 11:43:24.838377 28338 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 898e78341fba41619e23160907a0a1c9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 11:43:25.091912 28171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:25.086869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33568 (local address 127.24.76.125:37191)
0808 11:43:25.087138 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:25.087144 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:25.087407 (+ 263us) server_negotiation.cc:407] Connection header received
0808 11:43:25.087610 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:25.087614 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:25.087688 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:25.087766 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:25.088185 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:25.088965 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:25.090366 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:25.090785 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:25.090897 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:25.091244 (+ 347us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:25.091391 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:25.091460 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:25.091575 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 11:43:25.091642 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260808 11:43:25.102845 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:25.097576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54238 (local address 127.24.76.124:35599)
0808 11:43:25.097738 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:25.097743 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:25.098041 (+ 298us) server_negotiation.cc:407] Connection header received
0808 11:43:25.098212 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:25.098216 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:25.098421 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:25.098519 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:25.098842 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:25.099556 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:25.100833 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:25.101266 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:25.101352 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:25.101809 (+ 457us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:25.102373 (+ 564us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:25.102479 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:25.102576 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 11:43:25.102640 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 11:43:25.116001 28171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:25.108602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33582 (local address 127.24.76.125:37191)
0808 11:43:25.108780 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:25.108785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:25.109526 (+ 741us) server_negotiation.cc:407] Connection header received
0808 11:43:25.109970 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:25.109977 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:25.110071 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:25.110138 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:25.110881 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:25.112011 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:25.113976 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:25.114610 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:25.114696 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:25.115305 (+ 609us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:25.115403 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:25.115466 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:25.115612 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 11:43:25.115686 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 11:43:25.116407 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:25.108790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54254 (local address 127.24.76.124:35599)
0808 11:43:25.108957 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:25.108963 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:25.109933 (+ 970us) server_negotiation.cc:407] Connection header received
0808 11:43:25.110225 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:25.110231 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:25.110328 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:25.110387 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:25.111495 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:25.112613 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:25.114583 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:25.115149 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:25.115254 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:25.115715 (+ 461us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:25.115831 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:25.115916 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:25.116068 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 11:43:25.116142 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 11:43:25.120095 28618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:25.108599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50346 (local address 127.24.76.126:39265)
0808 11:43:25.112942 (+ 4343us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:25.112948 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:25.113381 (+ 433us) server_negotiation.cc:407] Connection header received
0808 11:43:25.113613 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:25.113618 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:25.113817 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:25.114125 (+ 308us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:25.115728 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:25.116849 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:25.118430 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:25.118822 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:25.118911 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:25.119279 (+ 368us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:25.119609 (+ 330us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:25.119692 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:25.119755 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 11:43:25.119814 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4226,"thread_start_us":4126,"threads_started":1}
I20260808 11:43:25.124454 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 27924
I20260808 11:43:25.125594 28026 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:43:25.239812 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 27924
I20260808 11:43:25.269687 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 28046
I20260808 11:43:25.270972 28150 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:43:25.379765 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 28046
I20260808 11:43:25.408416 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 28174
I20260808 11:43:25.409703 28285 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:43:25.517633 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 28174
2026-08-08T11:43:25Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4935 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 11:43:25 dist-test-slave-tc2s krb5kdc[28636](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 11:43:25 dist-test-slave-tc2s krb5kdc[28636](info): set up 2 sockets
Aug 08 11:43:25 dist-test-slave-tc2s krb5kdc[28636](info): commencing operation
krb5kdc: starting...
W20260808 11:43:26.771421 24881 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.002s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/krb5kdc/test-user.keytab.
Aug 08 11:43:26 dist-test-slave-tc2s krb5kdc[28636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T11:43:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T11:43:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.76.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
Entry for principal kudu/127.24.76.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab.
I20260808 11:43:27.083140 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:41893
--webserver_interface=127.24.76.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:45483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:41893,127.24.76.125:44731,127.24.76.124:37919
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 11:43:27.438373 28655 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 11:43:27.438683 28655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:43:27.438755 28655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:43:27.438836 28655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:43:27.445998 28655 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:43:27.446098 28655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:43:27.446161 28655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:43:27.446236 28655 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:43:27.446323 28655 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:43:27.456073 28655 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:45483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:41893,127.24.76.125:44731,127.24.76.124:37919
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.126.keytab
--principal=kudu/127.24.76.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.126:41893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.76.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:43:27.457372 28655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:43:27.458753 28655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:43:27.470086 28671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:27.470206 28666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:27.470120 28667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:43:27.471163 28655 server_base.cc:1034] running on GCE node
I20260808 11:43:27.471973 28655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:43:27.473325 28655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:43:27.474567 28655 hybrid_clock.cc:648] HybridClock initialized: now 1786189407474497 us; error 45 us; skew 500 ppm
I20260808 11:43:27.474884 28655 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:43:27 dist-test-slave-tc2s krb5kdc[28636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189407, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:43:27.477941 28655 init.cc:377] Logged in from keytab as kudu/127.24.76.126@KRBTEST.COM (short username kudu)
I20260808 11:43:27.479363 28655 webserver.cc:460] Webserver started at http://127.24.76.126:36917/ using document root <none> and password file <none>
I20260808 11:43:27.480074 28655 fs_manager.cc:359] Metadata directory not provided
I20260808 11:43:27.480181 28655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:43:27.480511 28655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:43:27.483542 28655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-0/data/instance:
uuid: "ad31a1cb568a4e2ba9c081be5fd36705"
format_stamp: "Formatted at 2026-08-08 11:43:27 on dist-test-slave-tc2s"
server_key: "cc94d7eb3f7ef3838d4b45523982974b"
server_key_iv: "8940d0a0c724bf7b0932ce776bf1d1f0"
server_key_version: "encryptionkey@0"
I20260808 11:43:27.484335 28655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-0/wal/instance:
uuid: "ad31a1cb568a4e2ba9c081be5fd36705"
format_stamp: "Formatted at 2026-08-08 11:43:27 on dist-test-slave-tc2s"
server_key: "cc94d7eb3f7ef3838d4b45523982974b"
server_key_iv: "8940d0a0c724bf7b0932ce776bf1d1f0"
server_key_version: "encryptionkey@0"
I20260808 11:43:27.489511 28655 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 11:43:27.493045 28682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:27.494498 28655 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 11:43:27.494654 28655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-0/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-0/wal
uuid: "ad31a1cb568a4e2ba9c081be5fd36705"
format_stamp: "Formatted at 2026-08-08 11:43:27 on dist-test-slave-tc2s"
server_key: "cc94d7eb3f7ef3838d4b45523982974b"
server_key_iv: "8940d0a0c724bf7b0932ce776bf1d1f0"
server_key_version: "encryptionkey@0"
I20260808 11:43:27.494855 28655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:43:27.518110 28655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:43:27.522405 28655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:43:27.522676 28655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:43:27.540477 28655 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.126:41893
I20260808 11:43:27.540508 28767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.126:41893 every 8 connection(s)
I20260808 11:43:27.541807 28655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-0/data/info.pb
I20260808 11:43:27.545637 28769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:27.548355 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 28655
I20260808 11:43:27.548564 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-0/wal/instance
I20260808 11:43:27.549129 24881 external_mini_cluster.cc:1428] Setting key e6befdc11554d9a9a7616f7813a8bd61
I20260808 11:43:27.553934 28769 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41893 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:43:27 dist-test-slave-tc2s krb5kdc[28636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189407, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:43:27.580065 28773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:27.557519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:41893 (local address 127.0.0.1:45492)
0808 11:43:27.558326 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:27.558378 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:27.560394 (+ 2016us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:27.561302 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:27.561307 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:27.561873 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:27.562255 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:27.562273 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:27.564283 (+ 2010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:27.564288 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:27.565537 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:27.565550 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:27.566474 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:27.566482 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:27.566660 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:27.570607 (+ 3947us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:27.570632 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:27.572841 (+ 2209us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:27.575947 (+ 3106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:27.575958 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:27.575972 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:27.576349 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:27.576742 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:27.576747 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:27.576751 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:27.576914 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:27.577660 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:27.577672 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:27.577876 (+ 204us) client_negotiation.cc:769] Sending connection context
0808 11:43:27.578137 (+ 261us) client_negotiation.cc:240] Negotiation successful
0808 11:43:27.578753 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"mutex_wait_us":129,"thread_start_us":341,"threads_started":1}
I20260808 11:43:27.580083 28777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:27.558782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45492 (local address 127.24.76.126:41893)
0808 11:43:27.559449 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:27.559457 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:27.559512 (+ 55us) server_negotiation.cc:407] Connection header received
0808 11:43:27.560773 (+ 1261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:27.560780 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:27.561083 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:27.561290 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:27.562488 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:27.564129 (+ 1641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:27.565668 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:27.566242 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:27.566408 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:27.573067 (+ 6659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:27.573076 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:27.573090 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:27.573119 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:27.575821 (+ 2702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:27.576480 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:27.576487 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:27.576489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:27.576637 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:27.577023 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:27.577026 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:27.577028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:27.577533 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:27.577725 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:27.578295 (+ 570us) server_negotiation.cc:299] Negotiation successful
0808 11:43:27.578585 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":301,"threads_started":1}
WARNING: no policy specified for kudu/127.24.76.125@KRBTEST.COM; defaulting to no policy
I20260808 11:43:27.583112 28769 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.76.125" port: 44731 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.125@KRBTEST.COM" created.
I20260808 11:43:27.584136 28773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:27.583749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:44731 (local address 127.0.0.1:49746)
0808 11:43:27.583911 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:27.584024 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.125:44731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 11:43:27.584384 28693 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.76.125:44731: connect: Connection refused (error 111)
I20260808 11:43:27.585990 28773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:27.585608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:44731 (local address 127.0.0.1:49750)
0808 11:43:27.585757 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:27.585825 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.125:44731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 11:43:27.586193 28769 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.125:44731: Network error: Client connection negotiation failed: client connection to 127.24.76.125:44731: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.125.keytab.
Entry for principal kudu/127.24.76.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.125.keytab.
I20260808 11:43:27.620855 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.125:44731
--webserver_interface=127.24.76.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.125.keytab
--principal=kudu/127.24.76.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:45483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:41893,127.24.76.125:44731,127.24.76.124:37919
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:43:27.635519 28769 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.125" port: 44731 } attempt: 1
I20260808 11:43:27.636755 28773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:27.636185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:44731 (local address 127.0.0.1:49752)
0808 11:43:27.636419 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:27.636579 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.125:44731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 11:43:27.637781 28773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:27.637398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:44731 (local address 127.0.0.1:49760)
0808 11:43:27.637545 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:27.637628 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.125:44731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 11:43:27.638028 28769 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.125:44731: Network error: Client connection negotiation failed: client connection to 127.24.76.125:44731: connect: Connection refused (error 111)
I20260808 11:43:27.706249 28769 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.125" port: 44731 } attempt: 2
I20260808 11:43:27.707510 28773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:27.706880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:44731 (local address 127.0.0.1:49766)
0808 11:43:27.707171 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:27.707294 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.125:44731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260808 11:43:27.708634 28773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:27.708197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:44731 (local address 127.0.0.1:49768)
0808 11:43:27.708350 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:27.708470 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.125:44731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 11:43:27.708819 28769 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.125:44731: Network error: Client connection negotiation failed: client connection to 127.24.76.125:44731: connect: Connection refused (error 111)
I20260808 11:43:27.800071 28769 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.125" port: 44731 } attempt: 3
I20260808 11:43:27.801229 28773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:27.800712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:44731 (local address 127.0.0.1:49780)
0808 11:43:27.800942 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:27.801062 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.125:44731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 11:43:27.802412 28773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:27.801979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:44731 (local address 127.0.0.1:49784)
0808 11:43:27.802138 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:27.802228 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.125:44731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 11:43:27.802671 28769 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.125:44731: Network error: Client connection negotiation failed: client connection to 127.24.76.125:44731: connect: Connection refused (error 111)
I20260808 11:43:27.945931 28769 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.125" port: 44731 } attempt: 4
I20260808 11:43:27.947184 28773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:27.946567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:44731 (local address 127.0.0.1:49792)
0808 11:43:27.946900 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:27.947012 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.125:44731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164}
I20260808 11:43:27.948279 28773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:27.947869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:44731 (local address 127.0.0.1:49796)
0808 11:43:27.948020 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:27.948124 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.125:44731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 11:43:27.948493 28769 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.125:44731: Network error: Client connection negotiation failed: client connection to 127.24.76.125:44731: connect: Connection refused (error 111)
W20260808 11:43:27.958907 28782 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 11:43:27.959192 28782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:43:27.959263 28782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:43:27.959344 28782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:43:27.967067 28782 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:43:27.967150 28782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:43:27.967213 28782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:43:27.967262 28782 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:43:27.967327 28782 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:43:27.977010 28782 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:45483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:41893,127.24.76.125:44731,127.24.76.124:37919
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.125.keytab
--principal=kudu/127.24.76.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.125:44731
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.76.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:43:27.978296 28782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:43:27.979552 28782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:43:27.990722 28793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:27.990746 28794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:27.990880 28797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:43:27.991621 28782 server_base.cc:1034] running on GCE node
I20260808 11:43:27.992365 28782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:43:27.993649 28782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:43:27.994890 28782 hybrid_clock.cc:648] HybridClock initialized: now 1786189407994821 us; error 44 us; skew 500 ppm
I20260808 11:43:27.995220 28782 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:43:27 dist-test-slave-tc2s krb5kdc[28636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189407, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:43:27.998413 28782 init.cc:377] Logged in from keytab as kudu/127.24.76.125@KRBTEST.COM (short username kudu)
I20260808 11:43:27.999852 28782 webserver.cc:460] Webserver started at http://127.24.76.125:33587/ using document root <none> and password file <none>
I20260808 11:43:28.000531 28782 fs_manager.cc:359] Metadata directory not provided
I20260808 11:43:28.000630 28782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:43:28.000933 28782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:43:28.003857 28782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-1/data/instance:
uuid: "d7be39e4e3c547be84b7b826be56e995"
format_stamp: "Formatted at 2026-08-08 11:43:27 on dist-test-slave-tc2s"
server_key: "d26df1bfe63a78eb162a80c0f10bc9c4"
server_key_iv: "ca11ba7539113866d0e7afb16a938743"
server_key_version: "encryptionkey@0"
I20260808 11:43:28.004622 28782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-1/wal/instance:
uuid: "d7be39e4e3c547be84b7b826be56e995"
format_stamp: "Formatted at 2026-08-08 11:43:27 on dist-test-slave-tc2s"
server_key: "d26df1bfe63a78eb162a80c0f10bc9c4"
server_key_iv: "ca11ba7539113866d0e7afb16a938743"
server_key_version: "encryptionkey@0"
I20260808 11:43:28.009357 28782 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 11:43:28.012755 28805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:28.014003 28782 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 11:43:28.014161 28782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-1/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-1/wal
uuid: "d7be39e4e3c547be84b7b826be56e995"
format_stamp: "Formatted at 2026-08-08 11:43:27 on dist-test-slave-tc2s"
server_key: "d26df1bfe63a78eb162a80c0f10bc9c4"
server_key_iv: "ca11ba7539113866d0e7afb16a938743"
server_key_version: "encryptionkey@0"
I20260808 11:43:28.014356 28782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:43:28.039714 28782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:43:28.043668 28782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:43:28.043907 28782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:43:28.060895 28782 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.125:44731
I20260808 11:43:28.060914 28889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.125:44731 every 8 connection(s)
I20260808 11:43:28.062233 28782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-1/data/info.pb
I20260808 11:43:28.063427 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 28782
I20260808 11:43:28.063616 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-1/wal/instance
I20260808 11:43:28.064189 24881 external_mini_cluster.cc:1428] Setting key f847db95cc1052c13c00aaeadb21e3ee
I20260808 11:43:28.066743 28891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:28.075141 28891 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41893 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:43:28 dist-test-slave-tc2s krb5kdc[28636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189407, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.125@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:43:28.096170 28777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.078242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45498 (local address 127.24.76.126:41893)
0808 11:43:28.078449 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:28.078456 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:28.079579 (+ 1123us) server_negotiation.cc:407] Connection header received
0808 11:43:28.081438 (+ 1859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:28.081442 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:28.081525 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:28.081601 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:28.083409 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.084091 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.086155 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.086570 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.086663 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:28.090593 (+ 3930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:28.090598 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:28.090601 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:28.090635 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:28.093114 (+ 2479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:28.093943 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:28.093950 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:28.093954 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:28.094087 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:28.094536 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:28.094539 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:28.094541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:28.094808 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:28.094918 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:28.095703 (+ 785us) server_negotiation.cc:299] Negotiation successful
0808 11:43:28.095939 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 11:43:28.097541 28895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.078654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:41893 (local address 127.0.0.1:45498)
0808 11:43:28.079390 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:28.079435 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:28.081151 (+ 1716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:28.081850 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:28.081860 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:28.082575 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:28.083252 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:28.083276 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.084253 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.084258 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:28.086027 (+ 1769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:28.086041 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.086798 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.086805 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:28.087056 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:28.087734 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:28.087768 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:28.090352 (+ 2584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:28.093251 (+ 2899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:28.093259 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:28.093274 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:28.093809 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:28.094200 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:28.094204 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:28.094208 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:28.094436 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:28.094928 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:28.094935 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:28.095323 (+ 388us) client_negotiation.cc:769] Sending connection context
0808 11:43:28.095635 (+ 312us) client_negotiation.cc:240] Negotiation successful
0808 11:43:28.096138 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":297,"threads_started":1}
I20260808 11:43:28.099310 28891 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.76.125" port: 44731 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.24.76.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.76.124@KRBTEST.COM" created.
Aug 08 11:43:28 dist-test-slave-tc2s krb5kdc[28636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189407, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.125@KRBTEST.COM for kudu/127.24.76.125@KRBTEST.COM
I20260808 11:43:28.112735 28895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.099959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:44731 (local address 127.0.0.1:49812)
0808 11:43:28.100213 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:28.100230 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:28.100352 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:28.101214 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:28.101218 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:28.101572 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:28.101708 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:28.101715 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.103216 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.103222 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:28.104124 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:28.104137 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.104882 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.104888 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:28.105068 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:28.105595 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:28.105610 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:28.107536 (+ 1926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:28.110375 (+ 2839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:28.110380 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:28.110384 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:28.110796 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:28.111156 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:28.111161 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:28.111165 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:28.111279 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:28.111840 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:28.111845 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:28.111978 (+ 133us) client_negotiation.cc:769] Sending connection context
0808 11:43:28.112094 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 11:43:28.112352 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151}
I20260808 11:43:28.112844 28897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.100184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49812 (local address 127.24.76.125:44731)
0808 11:43:28.100824 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:28.100830 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:28.100855 (+ 25us) server_negotiation.cc:407] Connection header received
0808 11:43:28.100897 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:28.100903 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:28.101072 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:28.101180 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:28.101856 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.103086 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.104258 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.104752 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.104861 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:28.107715 (+ 2854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:28.107726 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:28.107738 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:28.107769 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:28.110262 (+ 2493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:28.110910 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:28.110916 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:28.110919 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:28.111042 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:28.111374 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:28.111378 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:28.111380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:28.111704 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:28.111900 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:28.112227 (+ 327us) server_negotiation.cc:299] Negotiation successful
0808 11:43:28.112464 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":496,"thread_start_us":348,"threads_started":1}
I20260808 11:43:28.115031 28891 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.76.124" port: 37919 } has no permanent_uuid. Determining permanent_uuid...
I20260808 11:43:28.116110 28895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.115662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:37919 (local address 127.0.0.1:32950)
0808 11:43:28.115819 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:28.115955 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 11:43:28.116369 28810 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.76.124:37919: connect: Connection refused (error 111)
I20260808 11:43:28.117904 28895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.117526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:37919 (local address 127.0.0.1:32964)
0808 11:43:28.117664 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:28.117749 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 11:43:28.118131 28891 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.124:37919: Network error: Client connection negotiation failed: client connection to 127.24.76.124:37919: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.76.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.124.keytab.
Entry for principal kudu/127.24.76.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.124.keytab.
I20260808 11:43:28.140789 24881 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskryQpdG/build/asan/bin/kudu
/tmp/dist-test-taskryQpdG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.124:37919
--webserver_interface=127.24.76.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.124.keytab
--principal=kudu/127.24.76.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.76.84:45483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.76.126:41893,127.24.76.125:44731,127.24.76.124:37919
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 11:43:28.167481 28891 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.124" port: 37919 } attempt: 1
I20260808 11:43:28.168773 28895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.168214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:37919 (local address 127.0.0.1:32980)
0808 11:43:28.168453 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:28.168582 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260808 11:43:28.169863 28895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.169521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:37919 (local address 127.0.0.1:32982)
0808 11:43:28.169655 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:28.169748 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 11:43:28.170156 28891 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.124:37919: Network error: Client connection negotiation failed: client connection to 127.24.76.124:37919: connect: Connection refused (error 111)
I20260808 11:43:28.238371 28891 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.124" port: 37919 } attempt: 2
I20260808 11:43:28.239605 28895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.239162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:37919 (local address 127.0.0.1:32994)
0808 11:43:28.239351 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:28.239444 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260808 11:43:28.240594 28895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.240262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:37919 (local address 127.0.0.1:33006)
0808 11:43:28.240398 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:28.240470 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 11:43:28.240811 28891 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.124:37919: Network error: Client connection negotiation failed: client connection to 127.24.76.124:37919: connect: Connection refused (error 111)
I20260808 11:43:28.247715 28769 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.125" port: 44731 } attempt: 5
Aug 08 11:43:28 dist-test-slave-tc2s krb5kdc[28636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189407, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for kudu/127.24.76.125@KRBTEST.COM
I20260808 11:43:28.261785 28773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.248284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:44731 (local address 127.0.0.1:49820)
0808 11:43:28.248450 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:28.248467 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:28.248594 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:28.248959 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:28.248965 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:28.249351 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:28.249545 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:28.249556 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.250588 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.250593 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:28.251457 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:28.251470 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.252195 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.252201 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:28.252389 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:28.253396 (+ 1007us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:28.253420 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:28.255920 (+ 2500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:28.259541 (+ 3621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:28.259545 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:28.259548 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:28.259894 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:28.260304 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:28.260307 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:28.260310 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:28.260441 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:28.261051 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:28.261055 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:28.261177 (+ 122us) client_negotiation.cc:769] Sending connection context
0808 11:43:28.261307 (+ 130us) client_negotiation.cc:240] Negotiation successful
0808 11:43:28.261512 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 11:43:28.262020 28897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.248484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49820 (local address 127.24.76.125:44731)
0808 11:43:28.248652 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:28.248659 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:28.248678 (+ 19us) server_negotiation.cc:407] Connection header received
0808 11:43:28.248750 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:28.248754 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:28.248813 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:28.248895 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:28.249693 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.250444 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.251607 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.252050 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.252169 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:28.256122 (+ 3953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:28.256129 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:28.256133 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:28.256174 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:28.259414 (+ 3240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:28.260029 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:28.260034 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:28.260038 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:28.260195 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:28.260550 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:28.260554 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:28.260558 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:28.260953 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:28.261101 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:28.261440 (+ 339us) server_negotiation.cc:299] Negotiation successful
0808 11:43:28.261694 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 11:43:28.262915 28769 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.76.124" port: 37919 } has no permanent_uuid. Determining permanent_uuid...
I20260808 11:43:28.264083 28773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.263601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:37919 (local address 127.0.0.1:33016)
0808 11:43:28.263796 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:28.263907 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 11:43:28.265115 28773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.264746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:37919 (local address 127.0.0.1:33024)
0808 11:43:28.264891 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:28.264990 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 11:43:28.265355 28769 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.124:37919: Network error: Client connection negotiation failed: client connection to 127.24.76.124:37919: connect: Connection refused (error 111)
I20260808 11:43:28.316568 28769 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.124" port: 37919 } attempt: 1
I20260808 11:43:28.317775 28773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.317200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:37919 (local address 127.0.0.1:33030)
0808 11:43:28.317429 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:28.317570 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260808 11:43:28.318861 28773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.318476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:37919 (local address 127.0.0.1:33038)
0808 11:43:28.318633 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:28.318722 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 11:43:28.319097 28769 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.124:37919: Network error: Client connection negotiation failed: client connection to 127.24.76.124:37919: connect: Connection refused (error 111)
I20260808 11:43:28.332013 28891 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.124" port: 37919 } attempt: 3
I20260808 11:43:28.333058 28895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.332614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:37919 (local address 127.0.0.1:33044)
0808 11:43:28.332802 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:28.332887 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 11:43:28.334115 28895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.333651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:37919 (local address 127.0.0.1:33056)
0808 11:43:28.333867 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:28.333977 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
W20260808 11:43:28.334324 28891 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.124:37919: Network error: Client connection negotiation failed: client connection to 127.24.76.124:37919: connect: Connection refused (error 111)
I20260808 11:43:28.387295 28769 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.124" port: 37919 } attempt: 2
I20260808 11:43:28.388648 28773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.388066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:37919 (local address 127.0.0.1:33070)
0808 11:43:28.388306 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:28.388434 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 11:43:28.389811 28773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.389398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:37919 (local address 127.0.0.1:33084)
0808 11:43:28.389556 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:28.389665 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 11:43:28.390053 28769 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.124:37919: Network error: Client connection negotiation failed: client connection to 127.24.76.124:37919: connect: Connection refused (error 111)
I20260808 11:43:28.477514 28891 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.124" port: 37919 } attempt: 4
I20260808 11:43:28.478706 28895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.478240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:37919 (local address 127.0.0.1:33094)
0808 11:43:28.478454 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:28.478554 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 11:43:28.479609 28895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.479313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:37919 (local address 127.0.0.1:33102)
0808 11:43:28.479425 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:28.479504 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 11:43:28.479866 28891 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.124:37919: Network error: Client connection negotiation failed: client connection to 127.24.76.124:37919: connect: Connection refused (error 111)
W20260808 11:43:28.481606 28903 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 11:43:28.481935 28903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 11:43:28.482028 28903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 11:43:28.482136 28903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 11:43:28.489156 28903 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 11:43:28.489234 28903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 11:43:28.489303 28903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 11:43:28.489377 28903 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 11:43:28.489460 28903 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 11:43:28.496272 28769 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.124" port: 37919 } attempt: 3
I20260808 11:43:28.497303 28773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.496839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:37919 (local address 127.0.0.1:33104)
0808 11:43:28.497024 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:28.497121 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 11:43:28.498313 28773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.497872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:37919 (local address 127.0.0.1:33114)
0808 11:43:28.498101 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:28.498187 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.76.124:37919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
W20260808 11:43:28.498546 28769 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.76.124:37919: Network error: Client connection negotiation failed: client connection to 127.24.76.124:37919: connect: Connection refused (error 111)
I20260808 11:43:28.499704 28903 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.76.84:45483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.76.126:41893,127.24.76.125:44731,127.24.76.124:37919
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/krb5kdc/kudu_127.24.76.124.keytab
--principal=kudu/127.24.76.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.76.124:37919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.76.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 10:34:00 UTC on ccycloud.quasar-vgchxb.root.comops.site
build id 4574585
ASAN enabled
I20260808 11:43:28.500927 28903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 11:43:28.502182 28903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 11:43:28.513270 28908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:28.513329 28909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 11:43:28.513356 28911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 11:43:28.514135 28903 server_base.cc:1034] running on GCE node
I20260808 11:43:28.514797 28903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 11:43:28.516028 28903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 11:43:28.517230 28903 hybrid_clock.cc:648] HybridClock initialized: now 1786189408517175 us; error 25 us; skew 500 ppm
I20260808 11:43:28.517563 28903 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 11:43:28 dist-test-slave-tc2s krb5kdc[28636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189408, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 11:43:28.520591 28903 init.cc:377] Logged in from keytab as kudu/127.24.76.124@KRBTEST.COM (short username kudu)
I20260808 11:43:28.521883 28903 webserver.cc:460] Webserver started at http://127.24.76.124:45641/ using document root <none> and password file <none>
I20260808 11:43:28.522562 28903 fs_manager.cc:359] Metadata directory not provided
I20260808 11:43:28.522662 28903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 11:43:28.522959 28903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 11:43:28.526126 28903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-2/data/instance:
uuid: "3428f0b884a04ad4a6f196c6835af67e"
format_stamp: "Formatted at 2026-08-08 11:43:28 on dist-test-slave-tc2s"
server_key: "07973fed047863c41b86dffde66f3fe5"
server_key_iv: "a129e5d3b1df34765a300a5ea25c3b3c"
server_key_version: "encryptionkey@0"
I20260808 11:43:28.526885 28903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-2/wal/instance:
uuid: "3428f0b884a04ad4a6f196c6835af67e"
format_stamp: "Formatted at 2026-08-08 11:43:28 on dist-test-slave-tc2s"
server_key: "07973fed047863c41b86dffde66f3fe5"
server_key_iv: "a129e5d3b1df34765a300a5ea25c3b3c"
server_key_version: "encryptionkey@0"
I20260808 11:43:28.531674 28903 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 11:43:28.534864 28921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 11:43:28.536043 28903 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 11:43:28.536203 28903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-2/data,/tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-2/wal
uuid: "3428f0b884a04ad4a6f196c6835af67e"
format_stamp: "Formatted at 2026-08-08 11:43:28 on dist-test-slave-tc2s"
server_key: "07973fed047863c41b86dffde66f3fe5"
server_key_iv: "a129e5d3b1df34765a300a5ea25c3b3c"
server_key_version: "encryptionkey@0"
I20260808 11:43:28.536391 28903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 11:43:28.556645 28903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 11:43:28.560534 28903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 11:43:28.560775 28903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 11:43:28.577240 28903 rpc_server.cc:307] RPC server started. Bound to: 127.24.76.124:37919
I20260808 11:43:28.577261 29009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.76.124:37919 every 8 connection(s)
I20260808 11:43:28.578549 28903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-2/data/info.pb
I20260808 11:43:28.581346 24881 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskryQpdG/build/asan/bin/kudu as pid 28903
I20260808 11:43:28.581518 24881 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskryQpdG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786189042529715-24881-0/minicluster-data/master-2/wal/instance
I20260808 11:43:28.581966 24881 external_mini_cluster.cc:1428] Setting key 2dbd15c72e5249ee31acf5d7cc4515cf
I20260808 11:43:28.582242 29010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 11:43:28.591826 29010 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41893 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:43:28 dist-test-slave-tc2s krb5kdc[28636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.124@KRBTEST.COM
Aug 08 11:43:28 dist-test-slave-tc2s krb5kdc[28636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189408, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.124@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
I20260808 11:43:28.604929 29016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.583660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33128 (local address 127.24.76.124:37919)
0808 11:43:28.584441 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:28.584467 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:28.584517 (+ 50us) server_negotiation.cc:407] Connection header received
0808 11:43:28.586835 (+ 2318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:28.586862 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:28.587340 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:28.587778 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:28.589038 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.590997 (+ 1959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.592273 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.593268 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.593439 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:28.597576 (+ 4137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:28.597584 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:28.597594 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:28.597635 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:28.600927 (+ 3292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:28.601667 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:28.601673 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:28.601679 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:28.601830 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:28.602231 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:28.602236 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:28.602239 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:28.602850 (+ 611us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:28.603064 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:28.603581 (+ 517us) server_negotiation.cc:299] Negotiation successful
0808 11:43:28.603917 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":322,"threads_started":1}
I20260808 11:43:28.608322 29018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.594403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.126:41893 (local address 127.0.0.1:45504)
0808 11:43:28.595049 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:28.595089 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:28.595239 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:28.595611 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:28.595615 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:28.596179 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:28.596411 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:28.596419 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.597400 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.597403 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:28.598596 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:28.598612 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.599653 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.599658 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:28.599782 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:28.600333 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:28.600349 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:28.602535 (+ 2186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:28.605558 (+ 3023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:28.605568 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:28.605575 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:28.606126 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:28.606548 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:28.606551 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:28.606553 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:28.606670 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:28.607252 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:28.607259 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:28.607439 (+ 180us) client_negotiation.cc:769] Sending connection context
0808 11:43:28.607638 (+ 199us) client_negotiation.cc:240] Negotiation successful
0808 11:43:28.607921 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":271,"threads_started":1}
I20260808 11:43:28.608332 28777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.594522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45504 (local address 127.24.76.126:41893)
0808 11:43:28.594717 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:28.594723 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:28.595194 (+ 471us) server_negotiation.cc:407] Connection header received
0808 11:43:28.595361 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:28.595366 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:28.595455 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:28.595575 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:28.596557 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.597251 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.598821 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.599467 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.599600 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:28.602710 (+ 3110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:28.602715 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:28.602719 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:28.602753 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:28.605441 (+ 2688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:28.606255 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:28.606260 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:28.606263 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:28.606443 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:28.606773 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:28.606777 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:28.606781 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:28.607118 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:28.607611 (+ 493us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:28.607755 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 11:43:28.607959 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 11:43:28.609864 29010 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.76.125" port: 44731 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:43:28 dist-test-slave-tc2s krb5kdc[28636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.126@KRBTEST.COM
Aug 08 11:43:28 dist-test-slave-tc2s krb5kdc[28636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189408, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.124@KRBTEST.COM for kudu/127.24.76.125@KRBTEST.COM
I20260808 11:43:28.624363 28777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.607834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45520 (local address 127.24.76.126:41893)
0808 11:43:28.608548 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:28.608552 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:28.608566 (+ 14us) server_negotiation.cc:407] Connection header received
0808 11:43:28.608602 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:28.608605 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:28.608661 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:28.608748 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:28.610173 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.611798 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.613486 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.614181 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.614294 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:28.618418 (+ 4124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:28.618424 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:28.618429 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:28.618459 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:28.621768 (+ 3309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:28.622558 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:28.622562 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:28.622564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:28.622684 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:28.623057 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:28.623062 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:28.623065 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:28.623373 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:28.623643 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:28.623871 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 11:43:28.624034 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":14210,"threads_started":1}
I20260808 11:43:28.624625 29018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.610500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.125:44731 (local address 127.0.0.1:49824)
0808 11:43:28.610629 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:28.610645 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:28.610731 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:28.611061 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:28.611066 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:28.611400 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:28.611546 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:28.611557 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.612806 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.612811 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:28.613927 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:28.613942 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.615027 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.615037 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:28.615236 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:28.615795 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:28.615816 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:28.618715 (+ 2899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:28.622311 (+ 3596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:28.622316 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:28.622319 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:28.622722 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:28.623073 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:28.623076 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:28.623078 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:28.623205 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:28.623766 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:28.623770 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:28.623918 (+ 148us) client_negotiation.cc:769] Sending connection context
0808 11:43:28.624076 (+ 158us) client_negotiation.cc:240] Negotiation successful
0808 11:43:28.624316 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 11:43:28.625023 28897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.610630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49824 (local address 127.24.76.125:44731)
0808 11:43:28.610812 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:28.610817 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:28.610834 (+ 17us) server_negotiation.cc:407] Connection header received
0808 11:43:28.610885 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:28.610888 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:28.610944 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:28.611026 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:28.611682 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.612638 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.614061 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.614574 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.614698 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:28.618908 (+ 4210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:28.618914 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:28.618919 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:28.618959 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:28.622176 (+ 3217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:28.622833 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:28.622837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:28.622840 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:28.622965 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:28.623305 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:28.623310 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:28.623313 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:28.623645 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:28.623780 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:28.624186 (+ 406us) server_negotiation.cc:299] Negotiation successful
0808 11:43:28.624390 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 11:43:28.625923 29010 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.76.124" port: 37919 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 11:43:28 dist-test-slave-tc2s krb5kdc[28636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189408, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.124@KRBTEST.COM for kudu/127.24.76.124@KRBTEST.COM
Aug 08 11:43:28 dist-test-slave-tc2s krb5kdc[28636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.76.125@KRBTEST.COM
I20260808 11:43:28.638695 29018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.626655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:37919 (local address 127.0.0.1:33144)
0808 11:43:28.626847 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:28.626860 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:28.626935 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:28.627187 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:28.627190 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:28.627510 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:28.627627 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:28.627634 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.628584 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.628589 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:28.629461 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:28.629473 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.630272 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.630279 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:28.630476 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:28.630962 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:28.630984 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:28.632959 (+ 1975us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:28.636122 (+ 3163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:28.636127 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:28.636131 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:28.636610 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:28.636995 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:28.637000 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:28.637004 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:28.637157 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:28.637812 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:28.637815 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:28.638004 (+ 189us) client_negotiation.cc:769] Sending connection context
0808 11:43:28.638146 (+ 142us) client_negotiation.cc:240] Negotiation successful
0808 11:43:28.638435 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124}
I20260808 11:43:28.639134 29016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.626610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33144 (local address 127.24.76.124:37919)
0808 11:43:28.626781 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:28.626786 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:28.626917 (+ 131us) server_negotiation.cc:407] Connection header received
0808 11:43:28.627010 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:28.627014 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:28.627084 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:28.627181 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:28.627766 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.628452 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.629598 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.630125 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.630216 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:28.633140 (+ 2924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:28.633146 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:28.633151 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:28.633185 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:28.635995 (+ 2810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:28.636719 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:28.636724 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:28.636727 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:28.636857 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:28.637274 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:28.637279 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:28.637282 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:28.637702 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:28.637841 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:28.638239 (+ 398us) server_negotiation.cc:299] Negotiation successful
0808 11:43:28.638653 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":148,"server-negotiator.queue_time_us":55}
I20260808 11:43:28.640668 28897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.625995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49836 (local address 127.24.76.125:44731)
0808 11:43:28.626188 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:28.626194 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:28.626210 (+ 16us) server_negotiation.cc:407] Connection header received
0808 11:43:28.626304 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:28.626309 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:28.626367 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:28.626444 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:28.627911 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.628631 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.629859 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.630351 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.630460 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:28.634643 (+ 4183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:28.634648 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:28.634651 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:28.634679 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:28.637375 (+ 2696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:28.638135 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:28.638140 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:28.638143 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:28.638303 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:28.638726 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:28.638732 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:28.638736 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:28.639016 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:28.639219 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:28.639828 (+ 609us) server_negotiation.cc:299] Negotiation successful
0808 11:43:28.640334 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260808 11:43:28.644146 29010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3428f0b884a04ad4a6f196c6835af67e: Bootstrap starting.
I20260808 11:43:28.645179 24881 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 11:43:28.647876 29010 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3428f0b884a04ad4a6f196c6835af67e: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:28.648849 29010 log.cc:824] T 00000000000000000000000000000000 P 3428f0b884a04ad4a6f196c6835af67e: Log is configured to *not* fsync() on all Append() calls
I20260808 11:43:28.648080 28734 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ad31a1cb568a4e2ba9c081be5fd36705"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45520
I20260808 11:43:28.651461 29010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3428f0b884a04ad4a6f196c6835af67e: No bootstrap required, opened a new log
I20260808 11:43:28.658303 29010 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3428f0b884a04ad4a6f196c6835af67e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad31a1cb568a4e2ba9c081be5fd36705" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41893 } } peers { permanent_uuid: "d7be39e4e3c547be84b7b826be56e995" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 44731 } } peers { permanent_uuid: "3428f0b884a04ad4a6f196c6835af67e" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 37919 } }
I20260808 11:43:28.658584 29010 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3428f0b884a04ad4a6f196c6835af67e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3428f0b884a04ad4a6f196c6835af67e, State: Initialized, Role: FOLLOWER
I20260808 11:43:28.658983 29010 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3428f0b884a04ad4a6f196c6835af67e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad31a1cb568a4e2ba9c081be5fd36705" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41893 } } peers { permanent_uuid: "d7be39e4e3c547be84b7b826be56e995" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 44731 } } peers { permanent_uuid: "3428f0b884a04ad4a6f196c6835af67e" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 37919 } }
I20260808 11:43:28.659787 29023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3428f0b884a04ad4a6f196c6835af67e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad31a1cb568a4e2ba9c081be5fd36705" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41893 } } peers { permanent_uuid: "d7be39e4e3c547be84b7b826be56e995" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 44731 } } peers { permanent_uuid: "3428f0b884a04ad4a6f196c6835af67e" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 37919 } } }
I20260808 11:43:28.660096 29023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3428f0b884a04ad4a6f196c6835af67e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:43:28.660782 29010 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3428f0b884a04ad4a6f196c6835af67e [sys.catalog]: configured and running, proceeding with master startup.
W20260808 11:43:28.669541 29036 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3428f0b884a04ad4a6f196c6835af67e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 11:43:28.669703 29036 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 11:43:28.675760 28769 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.124" port: 37919 } attempt: 4
I20260808 11:43:28.667461 28903 master_runner.cc:186] Error getting master registration for 127.24.76.126:41893: OK, instance_id { permanent_uuid: "ad31a1cb568a4e2ba9c081be5fd36705" instance_seqno: 1786189407535576 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 11:43:28.678171 28903 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 08 11:43:28 dist-test-slave-tc2s krb5kdc[28636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189407, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.126@KRBTEST.COM for kudu/127.24.76.124@KRBTEST.COM
I20260808 11:43:28.691757 28773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.676346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:37919 (local address 127.0.0.1:33150)
0808 11:43:28.676580 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:28.676600 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:28.676710 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:28.677082 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:28.677087 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:28.677595 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:28.677797 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:28.677810 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.679286 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.679290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:28.680209 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:28.680221 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.681214 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.681220 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:28.681437 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:28.682146 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:28.682169 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:28.684984 (+ 2815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:28.688897 (+ 3913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:28.688903 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:28.688907 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:28.689430 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:28.689827 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:28.689832 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:28.689836 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:28.690067 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:28.690726 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:28.690732 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:28.690912 (+ 180us) client_negotiation.cc:769] Sending connection context
0808 11:43:28.691071 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 11:43:28.691359 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120}
I20260808 11:43:28.691767 29016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.676413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33150 (local address 127.24.76.124:37919)
0808 11:43:28.676612 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:28.676618 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:28.676647 (+ 29us) server_negotiation.cc:407] Connection header received
0808 11:43:28.676825 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:28.676831 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:28.676925 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:28.677056 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:28.677967 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.679134 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.680371 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.681075 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.681209 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:28.685184 (+ 3975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:28.685195 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:28.685200 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:28.685238 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:28.688753 (+ 3515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:28.689555 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:28.689561 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:28.689565 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:28.689713 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:28.690185 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:28.690190 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:28.690194 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:28.690579 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:28.690736 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:28.691202 (+ 466us) server_negotiation.cc:299] Negotiation successful
0808 11:43:28.691417 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 11:43:28.698179 28769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad31a1cb568a4e2ba9c081be5fd36705: Bootstrap starting.
I20260808 11:43:28.702258 28769 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad31a1cb568a4e2ba9c081be5fd36705: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:28.703579 28769 log.cc:824] T 00000000000000000000000000000000 P ad31a1cb568a4e2ba9c081be5fd36705: Log is configured to *not* fsync() on all Append() calls
I20260808 11:43:28.707026 28769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad31a1cb568a4e2ba9c081be5fd36705: No bootstrap required, opened a new log
I20260808 11:43:28.714903 28769 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad31a1cb568a4e2ba9c081be5fd36705 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad31a1cb568a4e2ba9c081be5fd36705" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41893 } } peers { permanent_uuid: "d7be39e4e3c547be84b7b826be56e995" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 44731 } } peers { permanent_uuid: "3428f0b884a04ad4a6f196c6835af67e" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 37919 } }
I20260808 11:43:28.715315 28769 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad31a1cb568a4e2ba9c081be5fd36705 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad31a1cb568a4e2ba9c081be5fd36705, State: Initialized, Role: FOLLOWER
I20260808 11:43:28.715811 28769 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad31a1cb568a4e2ba9c081be5fd36705 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad31a1cb568a4e2ba9c081be5fd36705" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41893 } } peers { permanent_uuid: "d7be39e4e3c547be84b7b826be56e995" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 44731 } } peers { permanent_uuid: "3428f0b884a04ad4a6f196c6835af67e" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 37919 } }
I20260808 11:43:28.716749 29040 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad31a1cb568a4e2ba9c081be5fd36705 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad31a1cb568a4e2ba9c081be5fd36705" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41893 } } peers { permanent_uuid: "d7be39e4e3c547be84b7b826be56e995" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 44731 } } peers { permanent_uuid: "3428f0b884a04ad4a6f196c6835af67e" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 37919 } } }
I20260808 11:43:28.717113 29040 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad31a1cb568a4e2ba9c081be5fd36705 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:43:28.718142 28769 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad31a1cb568a4e2ba9c081be5fd36705 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:43:28.718196 28734 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad31a1cb568a4e2ba9c081be5fd36705 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 11:43:28.718371 28734 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad31a1cb568a4e2ba9c081be5fd36705 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:28.721037 28734 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad31a1cb568a4e2ba9c081be5fd36705 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad31a1cb568a4e2ba9c081be5fd36705" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41893 } } peers { permanent_uuid: "d7be39e4e3c547be84b7b826be56e995" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 44731 } } peers { permanent_uuid: "3428f0b884a04ad4a6f196c6835af67e" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 37919 } }
I20260808 11:43:28.722241 28734 leader_election.cc:288] T 00000000000000000000000000000000 P ad31a1cb568a4e2ba9c081be5fd36705 [CANDIDATE]: Term 1 election: Requested vote from peers d7be39e4e3c547be84b7b826be56e995 (127.24.76.125:44731), 3428f0b884a04ad4a6f196c6835af67e (127.24.76.124:37919)
I20260808 11:43:28.723167 28972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ad31a1cb568a4e2ba9c081be5fd36705" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3428f0b884a04ad4a6f196c6835af67e"
I20260808 11:43:28.723670 28972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3428f0b884a04ad4a6f196c6835af67e [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:28.727108 28972 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3428f0b884a04ad4a6f196c6835af67e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad31a1cb568a4e2ba9c081be5fd36705 in term 1.
I20260808 11:43:28.727813 28691 leader_election.cc:302] T 00000000000000000000000000000000 P ad31a1cb568a4e2ba9c081be5fd36705 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3428f0b884a04ad4a6f196c6835af67e, ad31a1cb568a4e2ba9c081be5fd36705; no voters:
I20260808 11:43:28.728283 29040 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad31a1cb568a4e2ba9c081be5fd36705 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 11:43:28.741552 29040 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad31a1cb568a4e2ba9c081be5fd36705 [term 1 LEADER]: Becoming Leader. State: Replica: ad31a1cb568a4e2ba9c081be5fd36705, State: Running, Role: LEADER
I20260808 11:43:28.743114 29040 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad31a1cb568a4e2ba9c081be5fd36705 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad31a1cb568a4e2ba9c081be5fd36705" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41893 } } peers { permanent_uuid: "d7be39e4e3c547be84b7b826be56e995" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 44731 } } peers { permanent_uuid: "3428f0b884a04ad4a6f196c6835af67e" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 37919 } }
I20260808 11:43:28.757351 29050 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad31a1cb568a4e2ba9c081be5fd36705 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad31a1cb568a4e2ba9c081be5fd36705. Latest consensus state: current_term: 1 leader_uuid: "ad31a1cb568a4e2ba9c081be5fd36705" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad31a1cb568a4e2ba9c081be5fd36705" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41893 } } peers { permanent_uuid: "d7be39e4e3c547be84b7b826be56e995" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 44731 } } peers { permanent_uuid: "3428f0b884a04ad4a6f196c6835af67e" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 37919 } } }
I20260808 11:43:28.758014 29050 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad31a1cb568a4e2ba9c081be5fd36705 [sys.catalog]: This master's current role is: LEADER
I20260808 11:43:28.771221 29066 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260808 11:43:28.776449 28697 tablet.cc:2369] T 00000000000000000000000000000000 P ad31a1cb568a4e2ba9c081be5fd36705: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 11:43:28.779390 28891 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.76.124" port: 37919 } attempt: 5
I20260808 11:43:28.793162 29066 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 11:43:28.782768 28655 master_runner.cc:186] Error getting master registration for 127.24.76.125:44731: OK, instance_id { permanent_uuid: "d7be39e4e3c547be84b7b826be56e995" instance_seqno: 1786189408056040 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 11:43:28.801522 28655 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 11:43:28.803781 29016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.753038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33156 (local address 127.24.76.124:37919)
0808 11:43:28.753232 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:28.753239 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:28.762381 (+ 9142us) server_negotiation.cc:407] Connection header received
0808 11:43:28.763157 (+ 776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:28.763164 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:28.763256 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:28.763338 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:28.770771 (+ 7433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.771831 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.775791 (+ 3960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.776416 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.776527 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:28.787333 (+ 10806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:28.787341 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:28.787346 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:28.787390 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:28.791008 (+ 3618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:28.799389 (+ 8381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:28.799396 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:28.799401 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:28.799604 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:28.800079 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:28.800084 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:28.800088 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:28.800494 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:28.800742 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:28.803109 (+ 2367us) server_negotiation.cc:299] Negotiation successful
0808 11:43:28.803361 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 11:43:28.805768 28777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.756774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45524 (local address 127.24.76.126:41893)
0808 11:43:28.756965 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:28.756980 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:28.762755 (+ 5775us) server_negotiation.cc:407] Connection header received
0808 11:43:28.763565 (+ 810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:28.763572 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:28.763653 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:28.763720 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:28.767884 (+ 4164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.768969 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.775299 (+ 6330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.776141 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.776264 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:28.794749 (+ 18485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:28.794757 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:28.794762 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:28.794802 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:28.798527 (+ 3725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:28.800845 (+ 2318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:28.800851 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:28.800854 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:28.801044 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:28.801888 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:28.801912 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:28.801916 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:28.802290 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:28.802537 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:28.805144 (+ 2607us) server_negotiation.cc:299] Negotiation successful
0808 11:43:28.805395 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 11:43:28.812814 28932 tablet.cc:2369] T 00000000000000000000000000000000 P 3428f0b884a04ad4a6f196c6835af67e: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 11:43:28 dist-test-slave-tc2s krb5kdc[28636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786189407, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.76.125@KRBTEST.COM for kudu/127.24.76.124@KRBTEST.COM
I20260808 11:43:28.819350 28897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.774149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49850 (local address 127.24.76.125:44731)
0808 11:43:28.774335 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:28.774340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:28.785059 (+ 10719us) server_negotiation.cc:407] Connection header received
0808 11:43:28.785427 (+ 368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:28.785432 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:28.785545 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:28.785613 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:28.791831 (+ 6218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.792844 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.807516 (+ 14672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.808141 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.808234 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:28.812008 (+ 3774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:28.812016 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:28.812020 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:28.812061 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:28.815739 (+ 3678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:28.816534 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:28.816539 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:28.816542 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:28.816717 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:28.817162 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:28.817171 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:28.817174 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:28.817615 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:28.817857 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:28.818485 (+ 628us) server_negotiation.cc:299] Negotiation successful
0808 11:43:28.818892 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 11:43:28.821877 28972 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3428f0b884a04ad4a6f196c6835af67e [term 1 FOLLOWER]: Refusing update from remote peer ad31a1cb568a4e2ba9c081be5fd36705: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:43:28.827190 28895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.779962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.76.124:37919 (local address 127.0.0.1:33172)
0808 11:43:28.780344 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0808 11:43:28.780363 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 11:43:28.780466 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 11:43:28.800825 (+ 20359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 11:43:28.800831 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 11:43:28.801378 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0808 11:43:28.801588 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:28.801601 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.803342 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.803347 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:28.804758 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 11:43:28.804774 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.815454 (+ 10680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.815461 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 11:43:28.815720 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:28.816355 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 11:43:28.816378 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 11:43:28.820114 (+ 3736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 11:43:28.824452 (+ 4338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:28.824457 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:28.824460 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:28.824956 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:28.825407 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 11:43:28.825413 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 11:43:28.825417 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 11:43:28.825585 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 11:43:28.826259 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 11:43:28.826264 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 11:43:28.826417 (+ 153us) client_negotiation.cc:769] Sending connection context
0808 11:43:28.826567 (+ 150us) client_negotiation.cc:240] Negotiation successful
0808 11:43:28.826833 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235}
I20260808 11:43:28.828166 29075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:28.780664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33172 (local address 127.24.76.124:37919)
0808 11:43:28.800342 (+ 19678us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:28.800347 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:28.800371 (+ 24us) server_negotiation.cc:407] Connection header received
0808 11:43:28.800420 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:28.800425 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:28.800509 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:28.801746 (+ 1237us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:28.802008 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.803214 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.814014 (+ 10800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:28.815202 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:28.820307 (+ 5105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:28.820571 (+ 264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 11:43:28.820578 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 11:43:28.820583 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 11:43:28.820615 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 11:43:28.824296 (+ 3681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:28.825113 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:28.825118 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:28.825122 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:28.825275 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 11:43:28.825730 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 11:43:28.825735 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 11:43:28.825738 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 11:43:28.826112 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 11:43:28.827423 (+ 1311us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:28.827567 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 11:43:28.827773 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19517,"thread_start_us":19922,"threads_started":1}
I20260808 11:43:28.833760 28891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7be39e4e3c547be84b7b826be56e995: Bootstrap starting.
I20260808 11:43:28.834959 29050 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ad31a1cb568a4e2ba9c081be5fd36705 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3428f0b884a04ad4a6f196c6835af67e" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 37919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:43:28.843883 28891 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7be39e4e3c547be84b7b826be56e995: Neither blocks nor log segments found. Creating new log.
I20260808 11:43:28.845280 28891 log.cc:824] T 00000000000000000000000000000000 P d7be39e4e3c547be84b7b826be56e995: Log is configured to *not* fsync() on all Append() calls
I20260808 11:43:28.849341 28891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7be39e4e3c547be84b7b826be56e995: No bootstrap required, opened a new log
I20260808 11:43:28.855065 29023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3428f0b884a04ad4a6f196c6835af67e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad31a1cb568a4e2ba9c081be5fd36705. Latest consensus state: current_term: 1 leader_uuid: "ad31a1cb568a4e2ba9c081be5fd36705" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad31a1cb568a4e2ba9c081be5fd36705" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41893 } } peers { permanent_uuid: "d7be39e4e3c547be84b7b826be56e995" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 44731 } } peers { permanent_uuid: "3428f0b884a04ad4a6f196c6835af67e" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 37919 } } }
I20260808 11:43:28.855394 29023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3428f0b884a04ad4a6f196c6835af67e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:43:28.858568 29079 mvcc.cc:205] Tried to move back new op lower bound from 7316231818519564288 to 7316231818222333952. Current Snapshot: MvccSnapshot[applied={T|T < 7316231818519564288}]
I20260808 11:43:28.861402 28891 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7be39e4e3c547be84b7b826be56e995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad31a1cb568a4e2ba9c081be5fd36705" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41893 } } peers { permanent_uuid: "d7be39e4e3c547be84b7b826be56e995" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 44731 } } peers { permanent_uuid: "3428f0b884a04ad4a6f196c6835af67e" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 37919 } }
I20260808 11:43:28.861941 28891 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7be39e4e3c547be84b7b826be56e995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7be39e4e3c547be84b7b826be56e995, State: Initialized, Role: FOLLOWER
I20260808 11:43:28.862561 28891 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7be39e4e3c547be84b7b826be56e995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad31a1cb568a4e2ba9c081be5fd36705" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41893 } } peers { permanent_uuid: "d7be39e4e3c547be84b7b826be56e995" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 44731 } } peers { permanent_uuid: "3428f0b884a04ad4a6f196c6835af67e" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 37919 } }
I20260808 11:43:28.862525 29040 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad31a1cb568a4e2ba9c081be5fd36705 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ad31a1cb568a4e2ba9c081be5fd36705" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad31a1cb568a4e2ba9c081be5fd36705" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41893 } } peers { permanent_uuid: "d7be39e4e3c547be84b7b826be56e995" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 44731 } } peers { permanent_uuid: "3428f0b884a04ad4a6f196c6835af67e" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 37919 } } }
I20260808 11:43:28.862867 29040 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad31a1cb568a4e2ba9c081be5fd36705 [sys.catalog]: This master's current role is: LEADER
I20260808 11:43:28.870718 29085 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7be39e4e3c547be84b7b826be56e995 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad31a1cb568a4e2ba9c081be5fd36705" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41893 } } peers { permanent_uuid: "d7be39e4e3c547be84b7b826be56e995" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 44731 } } peers { permanent_uuid: "3428f0b884a04ad4a6f196c6835af67e" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 37919 } } }
I20260808 11:43:28.871029 29023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3428f0b884a04ad4a6f196c6835af67e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ad31a1cb568a4e2ba9c081be5fd36705" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad31a1cb568a4e2ba9c081be5fd36705" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41893 } } peers { permanent_uuid: "d7be39e4e3c547be84b7b826be56e995" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 44731 } } peers { permanent_uuid: "3428f0b884a04ad4a6f196c6835af67e" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 37919 } } }
I20260808 11:43:28.871263 29023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3428f0b884a04ad4a6f196c6835af67e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:43:28.872982 29085 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7be39e4e3c547be84b7b826be56e995 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:43:28.872859 28891 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7be39e4e3c547be84b7b826be56e995 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 11:43:28.873215 28848 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7be39e4e3c547be84b7b826be56e995 [term 0 FOLLOWER]: Advancing to term 1
I20260808 11:43:28.874037 28851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ad31a1cb568a4e2ba9c081be5fd36705" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d7be39e4e3c547be84b7b826be56e995"
I20260808 11:43:28.876374 29066 catalog_manager.cc:1347] Generated new cluster ID: 1413b8ed9eec4927a20c79667b292dda
I20260808 11:43:28.876480 29066 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 11:43:28.877663 28848 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d7be39e4e3c547be84b7b826be56e995 [term 1 FOLLOWER]: Refusing update from remote peer ad31a1cb568a4e2ba9c081be5fd36705: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 11:43:28.884598 28851 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d7be39e4e3c547be84b7b826be56e995 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad31a1cb568a4e2ba9c081be5fd36705 in term 1.
I20260808 11:43:28.886826 29050 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ad31a1cb568a4e2ba9c081be5fd36705 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7be39e4e3c547be84b7b826be56e995" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 44731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 11:43:28.902654 29066 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 11:43:28.904446 29066 catalog_manager.cc:1504] Loading token signing keys...
I20260808 11:43:28.921933 29066 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad31a1cb568a4e2ba9c081be5fd36705: Generated new TSK 0
I20260808 11:43:28.922909 29066 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 11:43:28.922744 29085 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7be39e4e3c547be84b7b826be56e995 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad31a1cb568a4e2ba9c081be5fd36705. Latest consensus state: current_term: 1 leader_uuid: "ad31a1cb568a4e2ba9c081be5fd36705" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad31a1cb568a4e2ba9c081be5fd36705" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41893 } } peers { permanent_uuid: "d7be39e4e3c547be84b7b826be56e995" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 44731 } } peers { permanent_uuid: "3428f0b884a04ad4a6f196c6835af67e" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 37919 } } }
I20260808 11:43:28.923041 29085 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7be39e4e3c547be84b7b826be56e995 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:43:28.924299 29050 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad31a1cb568a4e2ba9c081be5fd36705 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ad31a1cb568a4e2ba9c081be5fd36705" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad31a1cb568a4e2ba9c081be5fd36705" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41893 } } peers { permanent_uuid: "d7be39e4e3c547be84b7b826be56e995" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 44731 } } peers { permanent_uuid: "3428f0b884a04ad4a6f196c6835af67e" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 37919 } } }
I20260808 11:43:28.924511 29050 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad31a1cb568a4e2ba9c081be5fd36705 [sys.catalog]: This master's current role is: LEADER
I20260808 11:43:28.925081 29095 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7be39e4e3c547be84b7b826be56e995 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ad31a1cb568a4e2ba9c081be5fd36705" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad31a1cb568a4e2ba9c081be5fd36705" member_type: VOTER last_known_addr { host: "127.24.76.126" port: 41893 } } peers { permanent_uuid: "d7be39e4e3c547be84b7b826be56e995" member_type: VOTER last_known_addr { host: "127.24.76.125" port: 44731 } } peers { permanent_uuid: "3428f0b884a04ad4a6f196c6835af67e" member_type: VOTER last_known_addr { host: "127.24.76.124" port: 37919 } } }
I20260808 11:43:28.925420 29095 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7be39e4e3c547be84b7b826be56e995 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 11:43:28.981374 29108 catalog_manager.cc:1259] Loaded cluster ID: 1413b8ed9eec4927a20c79667b292dda
I20260808 11:43:28.981505 29108 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d7be39e4e3c547be84b7b826be56e995: loading cluster ID for follower catalog manager: success
I20260808 11:43:28.987720 29108 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d7be39e4e3c547be84b7b826be56e995: acquiring CA information for follower catalog manager: success
I20260808 11:43:28.990504 29108 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d7be39e4e3c547be84b7b826be56e995: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 11:43:29.074093 28777 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:43:29.065057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45538 (local address 127.24.76.126:41893)
0808 11:43:29.065220 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:29.065226 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:29.071071 (+ 5845us) server_negotiation.cc:407] Connection header received
0808 11:43:29.072489 (+ 1418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:29.072496 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:29.072600 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:29.072675 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:29.073802 (+ 1127us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45538: BlockingRecv error: recv got EOF from 127.0.0.1:45538 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260808 11:43:29.075845 29075 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:43:29.061574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33180 (local address 127.24.76.124:37919)
0808 11:43:29.062054 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:29.062061 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:29.070610 (+ 8549us) server_negotiation.cc:407] Connection header received
0808 11:43:29.074421 (+ 3811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:29.074431 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:29.074536 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:29.074600 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:29.075614 (+ 1014us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33180: BlockingRecv error: recv got EOF from 127.0.0.1:33180 (error 108)
Metrics: {"server-negotiator.queue_time_us":342}
W20260808 11:43:29.078239 28897 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:43:29.071574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49856 (local address 127.24.76.125:44731)
0808 11:43:29.071764 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:29.071770 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:29.076631 (+ 4861us) server_negotiation.cc:407] Connection header received
0808 11:43:29.076943 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:29.076948 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:29.077044 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:29.077114 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:29.078029 (+ 915us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49856: BlockingRecv error: recv got EOF from 127.0.0.1:49856 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 11:43:29.134509 28897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:29.119039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49872 (local address 127.24.76.125:44731)
0808 11:43:29.119238 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:29.119244 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:29.126505 (+ 7261us) server_negotiation.cc:407] Connection header received
0808 11:43:29.127037 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:29.127044 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:29.127251 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:29.127389 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:29.128133 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:29.129289 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:29.131302 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:29.132012 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:29.132117 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:29.132892 (+ 775us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:29.133623 (+ 731us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:29.133778 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:29.134061 (+ 283us) server_negotiation.cc:299] Negotiation successful
0808 11:43:29.134157 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260808 11:43:29.177470 29075 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:43:29.167800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33190 (local address 127.24.76.124:37919)
0808 11:43:29.167987 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:29.167993 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:29.175325 (+ 7332us) server_negotiation.cc:407] Connection header received
0808 11:43:29.175844 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:29.175850 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:29.176013 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:29.176101 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:29.177189 (+ 1088us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33190: BlockingRecv error: recv got EOF from 127.0.0.1:33190 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 11:43:29.244935 28897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:29.227741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49882 (local address 127.24.76.125:44731)
0808 11:43:29.227975 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:29.227983 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:29.239945 (+ 11962us) server_negotiation.cc:407] Connection header received
0808 11:43:29.240168 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:29.240174 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:29.240273 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:29.240369 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:29.241003 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:29.241751 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:29.243157 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:29.243642 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:29.243758 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:29.244178 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:29.244291 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:29.244370 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:29.244531 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 11:43:29.244617 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260808 11:43:29.253333 29075 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:43:29.251307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33194 (local address 127.24.76.124:37919)
0808 11:43:29.251515 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:29.251522 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:29.252011 (+ 489us) server_negotiation.cc:407] Connection header received
0808 11:43:29.252195 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:29.252201 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:29.252280 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:29.252371 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:29.253161 (+ 790us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33194: BlockingRecv error: recv got EOF from 127.0.0.1:33194 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 11:43:29.266849 28897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:29.261272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49894 (local address 127.24.76.125:44731)
0808 11:43:29.261450 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:29.261456 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:29.261966 (+ 510us) server_negotiation.cc:407] Connection header received
0808 11:43:29.262281 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:29.262287 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:29.262367 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:29.262444 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:29.262973 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:29.263825 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:29.265100 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:29.265560 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:29.265666 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:29.266108 (+ 442us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:29.266222 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:29.266299 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:29.266454 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 11:43:29.266542 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260808 11:43:29.277089 29075 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:43:29.274988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33198 (local address 127.24.76.124:37919)
0808 11:43:29.275192 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:29.275199 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:29.275734 (+ 535us) server_negotiation.cc:407] Connection header received
0808 11:43:29.275889 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:29.275895 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:29.275989 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:29.276073 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:29.276866 (+ 793us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33198: BlockingRecv error: recv got EOF from 127.0.0.1:33198 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 11:43:29.289222 28897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:29.283915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49896 (local address 127.24.76.125:44731)
0808 11:43:29.284156 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:29.284162 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:29.284742 (+ 580us) server_negotiation.cc:407] Connection header received
0808 11:43:29.284930 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:29.284936 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:29.285005 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:29.285077 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:29.285474 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:29.286256 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:29.287579 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:29.288047 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:29.288155 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:29.288528 (+ 373us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:29.288633 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:29.288706 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:29.288844 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 11:43:29.288924 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
W20260808 11:43:29.296846 29075 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:43:29.294757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33210 (local address 127.24.76.124:37919)
0808 11:43:29.295032 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:29.295038 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:29.295509 (+ 471us) server_negotiation.cc:407] Connection header received
0808 11:43:29.295649 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:29.295655 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:29.295731 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:29.295820 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:29.296577 (+ 757us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33210: BlockingRecv error: recv got EOF from 127.0.0.1:33210 (error 108)
Metrics: {"server-negotiator.queue_time_us":153}
I20260808 11:43:29.313125 28897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:29.307593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49898 (local address 127.24.76.125:44731)
0808 11:43:29.307797 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:29.307803 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:29.308294 (+ 491us) server_negotiation.cc:407] Connection header received
0808 11:43:29.308479 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:29.308483 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:29.308551 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:29.308626 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:29.309044 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:29.309977 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:29.311242 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:29.311708 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:29.311830 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:29.312245 (+ 415us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:29.312367 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:29.312443 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:29.312725 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 11:43:29.312806 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260808 11:43:29.323459 29075 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:43:29.321438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33220 (local address 127.24.76.124:37919)
0808 11:43:29.321655 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:29.321662 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:29.322144 (+ 482us) server_negotiation.cc:407] Connection header received
0808 11:43:29.322307 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:29.322313 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:29.322387 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:29.322470 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:29.323232 (+ 762us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33220: BlockingRecv error: recv got EOF from 127.0.0.1:33220 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 11:43:29.348589 28897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:29.343205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49904 (local address 127.24.76.125:44731)
0808 11:43:29.343397 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:29.343403 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:29.343933 (+ 530us) server_negotiation.cc:407] Connection header received
0808 11:43:29.344127 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:29.344131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:29.344200 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:29.344278 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:29.344713 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:29.345521 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:29.346840 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:29.347306 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:29.347427 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:29.347896 (+ 469us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:29.347999 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:29.348086 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:29.348229 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 11:43:29.348307 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260808 11:43:29.357667 29075 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:43:29.355852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33236 (local address 127.24.76.124:37919)
0808 11:43:29.356013 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:29.356020 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:29.356397 (+ 377us) server_negotiation.cc:407] Connection header received
0808 11:43:29.356562 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:29.356568 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:29.356648 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:29.356730 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:29.357486 (+ 756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33236: BlockingRecv error: recv got EOF from 127.0.0.1:33236 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 11:43:29.399112 28897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:29.393462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49920 (local address 127.24.76.125:44731)
0808 11:43:29.393760 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:29.393768 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:29.394321 (+ 553us) server_negotiation.cc:407] Connection header received
0808 11:43:29.394532 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:29.394540 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:29.394619 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:29.394701 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:29.395159 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:29.395899 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:29.397233 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:29.397751 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:29.397884 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:29.398338 (+ 454us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:29.398449 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:29.398532 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:29.398677 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 11:43:29.398758 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260808 11:43:29.414791 29075 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:43:29.412597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33240 (local address 127.24.76.124:37919)
0808 11:43:29.412796 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:29.412803 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:29.413330 (+ 527us) server_negotiation.cc:407] Connection header received
0808 11:43:29.413545 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:29.413551 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:29.413628 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:29.413714 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:29.414552 (+ 838us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33240: BlockingRecv error: recv got EOF from 127.0.0.1:33240 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 11:43:29.491472 28897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:29.486652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49926 (local address 127.24.76.125:44731)
0808 11:43:29.486886 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:29.486893 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:29.487404 (+ 511us) server_negotiation.cc:407] Connection header received
0808 11:43:29.487588 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:29.487595 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:29.487670 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:29.487764 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:29.488133 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:29.488851 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:29.490148 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:29.490547 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:29.490638 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:29.490939 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:29.491044 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:29.491104 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:29.491198 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 11:43:29.491261 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260808 11:43:29.498724 29075 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:43:29.496906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33254 (local address 127.24.76.124:37919)
0808 11:43:29.497100 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:29.497106 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:29.497432 (+ 326us) server_negotiation.cc:407] Connection header received
0808 11:43:29.497602 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:29.497606 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:29.497680 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:29.497777 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:29.498513 (+ 736us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33254: BlockingRecv error: recv got EOF from 127.0.0.1:33254 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 11:43:29.635389 28897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:29.630379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49940 (local address 127.24.76.125:44731)
0808 11:43:29.630555 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:29.630559 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:29.631064 (+ 505us) server_negotiation.cc:407] Connection header received
0808 11:43:29.631240 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:29.631245 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:29.631316 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:29.631397 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:29.631737 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:29.632430 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:29.633842 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:29.634276 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:29.634368 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:29.634812 (+ 444us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:29.634913 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:29.634987 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:29.635121 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 11:43:29.635183 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260808 11:43:29.642515 29075 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 11:43:29.640813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33262 (local address 127.24.76.124:37919)
0808 11:43:29.640988 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:29.640992 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:29.641318 (+ 326us) server_negotiation.cc:407] Connection header received
0808 11:43:29.641454 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:29.641458 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:29.641527 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:29.641607 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 11:43:29.642317 (+ 710us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33262: BlockingRecv error: recv got EOF from 127.0.0.1:33262 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 11:43:29.671665 29036 catalog_manager.cc:1259] Loaded cluster ID: 1413b8ed9eec4927a20c79667b292dda
I20260808 11:43:29.671758 29036 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3428f0b884a04ad4a6f196c6835af67e: loading cluster ID for follower catalog manager: success
I20260808 11:43:29.674357 29036 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3428f0b884a04ad4a6f196c6835af67e: acquiring CA information for follower catalog manager: success
I20260808 11:43:29.676115 29036 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3428f0b884a04ad4a6f196c6835af67e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 11:43:29.908039 28897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:29.903610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49942 (local address 127.24.76.125:44731)
0808 11:43:29.903805 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:29.903809 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:29.904013 (+ 204us) server_negotiation.cc:407] Connection header received
0808 11:43:29.904184 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:29.904188 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:29.904257 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:29.904341 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:29.904701 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:29.905415 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:29.906685 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:29.907092 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:29.907177 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:29.907501 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:29.907604 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:29.907661 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:29.907761 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 11:43:29.907830 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 11:43:29.919158 29075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:29.913941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33264 (local address 127.24.76.124:37919)
0808 11:43:29.914160 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:29.914166 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:29.914356 (+ 190us) server_negotiation.cc:407] Connection header received
0808 11:43:29.914468 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:29.914472 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:29.914634 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:29.914735 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:29.915069 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:29.915802 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:29.917044 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:29.917471 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:29.917556 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:29.918054 (+ 498us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:29.918624 (+ 570us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:29.918743 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:29.918813 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 11:43:29.918880 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 11:43:29.931205 29075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:29.924699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33274 (local address 127.24.76.124:37919)
0808 11:43:29.924853 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:29.924859 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:29.925758 (+ 899us) server_negotiation.cc:407] Connection header received
0808 11:43:29.926027 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:29.926034 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:29.926114 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:29.926178 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:29.926829 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:29.927699 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:29.929395 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:29.929920 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:29.930039 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:29.930540 (+ 501us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:29.930648 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:29.930720 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:29.930865 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 11:43:29.930935 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 11:43:29.932797 29306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:29.924622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45548 (local address 127.24.76.126:41893)
0808 11:43:29.925212 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:29.925218 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:29.925613 (+ 395us) server_negotiation.cc:407] Connection header received
0808 11:43:29.925913 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:29.925920 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:29.926160 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:29.926308 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:29.927511 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:29.928467 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:29.930455 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:29.931067 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:29.931183 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:29.931811 (+ 628us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:29.932247 (+ 436us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:29.932355 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:29.932509 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 11:43:29.932591 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":353,"threads_started":1}
I20260808 11:43:29.933633 28897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 11:43:29.925616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49948 (local address 127.24.76.125:44731)
0808 11:43:29.925780 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 11:43:29.925785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 11:43:29.926469 (+ 684us) server_negotiation.cc:407] Connection header received
0808 11:43:29.926654 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 11:43:29.926659 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 11:43:29.926737 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 11:43:29.926835 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 11:43:29.928485 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:29.929528 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:29.931531 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 11:43:29.932009 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 11:43:29.932096 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 11:43:29.932905 (+ 809us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 11:43:29.933044 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 11:43:29.933110 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 11:43:29.933270 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 11:43:29.933345 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 11:43:29.937707 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 28655
I20260808 11:43:29.938725 28761 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:43:30.047257 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 28655
I20260808 11:43:30.075663 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 28782
I20260808 11:43:30.076726 28885 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:43:30.182608 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 28782
I20260808 11:43:30.212653 24881 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskryQpdG/build/asan/bin/kudu pid 28903
I20260808 11:43:30.213766 29003 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 11:43:30.322129 24881 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskryQpdG/build/asan/bin/kudu with pid 28903
2026-08-08T11:43:30Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4803 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9738 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (367683 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST